2019-01-28 16:47:54.069 23745 INFO sysinv.agent.lldp.manager [-] Configured sysinv LLDP agent drivers: [] 2019-01-28 16:47:54.104 23745 INFO sysinv.agent.lldp.manager [-] Loaded sysinv LLDP agent drivers: [] 2019-01-28 16:47:54.104 23745 INFO sysinv.agent.lldp.manager [-] Registered sysinv LLDP agent drivers: [] 2019-01-28 16:47:54.143 23745 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2019-01-28 16:47:55.144 23745 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on localhost:5672 2019-01-28 16:47:55.150 23745 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds. 2019-01-28 16:47:58.154 23745 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on localhost:5672 2019-01-28 16:47:58.161 23745 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds. 2019-01-28 16:48:03.164 23745 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on localhost:5672 2019-01-28 16:48:03.190 23745 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on localhost:5672 2019-01-28 16:48:03.209 23745 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 16:48:03.734 23745 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 16:48:03.737 23745 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 16:48:03.738 23745 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 16:48:03.970 23745 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 16:48:03.973 23745 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 16:48:03.973 23745 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 16:48:04.208 23745 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 16:48:04.210 23745 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 16:48:04.211 23745 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 16:48:04.233 23745 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on localhost:5672 2019-01-28 16:48:08.517 29105 INFO migrate.versioning.api [-] 0 -> 1... 2019-01-28 16:48:08.816 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:08.816 29105 INFO migrate.versioning.api [-] 1 -> 2... 2019-01-28 16:48:09.584 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.584 29105 INFO migrate.versioning.api [-] 2 -> 3... 2019-01-28 16:48:09.587 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.587 29105 INFO migrate.versioning.api [-] 3 -> 4... 2019-01-28 16:48:09.590 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.590 29105 INFO migrate.versioning.api [-] 4 -> 5... 2019-01-28 16:48:09.594 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.594 29105 INFO migrate.versioning.api [-] 5 -> 6... 2019-01-28 16:48:09.597 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.597 29105 INFO migrate.versioning.api [-] 6 -> 7... 2019-01-28 16:48:09.600 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.600 29105 INFO migrate.versioning.api [-] 7 -> 8... 2019-01-28 16:48:09.604 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.604 29105 INFO migrate.versioning.api [-] 8 -> 9... 2019-01-28 16:48:09.607 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.607 29105 INFO migrate.versioning.api [-] 9 -> 10... 2019-01-28 16:48:09.610 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.611 29105 INFO migrate.versioning.api [-] 10 -> 11... 2019-01-28 16:48:09.614 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.614 29105 INFO migrate.versioning.api [-] 11 -> 12... 2019-01-28 16:48:09.617 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.617 29105 INFO migrate.versioning.api [-] 12 -> 13... 2019-01-28 16:48:09.621 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.621 29105 INFO migrate.versioning.api [-] 13 -> 14... 2019-01-28 16:48:09.625 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.625 29105 INFO migrate.versioning.api [-] 14 -> 15... 2019-01-28 16:48:09.628 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.628 29105 INFO migrate.versioning.api [-] 15 -> 16... 2019-01-28 16:48:09.631 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.632 29105 INFO migrate.versioning.api [-] 16 -> 17... 2019-01-28 16:48:09.635 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.635 29105 INFO migrate.versioning.api [-] 17 -> 18... 2019-01-28 16:48:09.638 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.639 29105 INFO migrate.versioning.api [-] 18 -> 19... 2019-01-28 16:48:09.642 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.642 29105 INFO migrate.versioning.api [-] 19 -> 20... 2019-01-28 16:48:09.644 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.645 29105 INFO migrate.versioning.api [-] 20 -> 21... 2019-01-28 16:48:09.648 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.648 29105 INFO migrate.versioning.api [-] 21 -> 22... 2019-01-28 16:48:09.650 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.651 29105 INFO migrate.versioning.api [-] 22 -> 23... 2019-01-28 16:48:09.653 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.654 29105 INFO migrate.versioning.api [-] 23 -> 24... 2019-01-28 16:48:09.657 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.657 29105 INFO migrate.versioning.api [-] 24 -> 25... 2019-01-28 16:48:09.660 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.661 29105 INFO migrate.versioning.api [-] 25 -> 26... 2019-01-28 16:48:09.664 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.664 29105 INFO migrate.versioning.api [-] 26 -> 27... 2019-01-28 16:48:09.667 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.667 29105 INFO migrate.versioning.api [-] 27 -> 28... 2019-01-28 16:48:09.670 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.670 29105 INFO migrate.versioning.api [-] 28 -> 29... 2019-01-28 16:48:09.673 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.673 29105 INFO migrate.versioning.api [-] 29 -> 30... 2019-01-28 16:48:09.676 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.676 29105 INFO migrate.versioning.api [-] 30 -> 31... 2019-01-28 16:48:09.697 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.697 29105 INFO migrate.versioning.api [-] 31 -> 32... 2019-01-28 16:48:09.709 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.709 29105 INFO migrate.versioning.api [-] 32 -> 33... 2019-01-28 16:48:09.728 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.728 29105 INFO migrate.versioning.api [-] 33 -> 34... 2019-01-28 16:48:09.778 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.778 29105 INFO migrate.versioning.api [-] 34 -> 35... 2019-01-28 16:48:09.791 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.791 29105 INFO migrate.versioning.api [-] 35 -> 36... 2019-01-28 16:48:09.871 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.871 29105 INFO migrate.versioning.api [-] 36 -> 37... 2019-01-28 16:48:09.897 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.897 29105 INFO migrate.versioning.api [-] 37 -> 38... 2019-01-28 16:48:09.968 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:09.969 29105 INFO migrate.versioning.api [-] 38 -> 39... 2019-01-28 16:48:10.032 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.033 29105 INFO migrate.versioning.api [-] 39 -> 40... 2019-01-28 16:48:10.053 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.054 29105 INFO migrate.versioning.api [-] 40 -> 41... 2019-01-28 16:48:10.101 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.101 29105 INFO migrate.versioning.api [-] 41 -> 42... 2019-01-28 16:48:10.143 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.143 29105 INFO migrate.versioning.api [-] 42 -> 43... 2019-01-28 16:48:10.213 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.213 29105 INFO migrate.versioning.api [-] 43 -> 44... 2019-01-28 16:48:10.350 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.351 29105 INFO migrate.versioning.api [-] 44 -> 45... 2019-01-28 16:48:10.402 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.402 29105 INFO migrate.versioning.api [-] 45 -> 46... 2019-01-28 16:48:10.405 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.405 29105 INFO migrate.versioning.api [-] 46 -> 47... 2019-01-28 16:48:10.408 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.408 29105 INFO migrate.versioning.api [-] 47 -> 48... 2019-01-28 16:48:10.411 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.411 29105 INFO migrate.versioning.api [-] 48 -> 49... 2019-01-28 16:48:10.414 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.414 29105 INFO migrate.versioning.api [-] 49 -> 50... 2019-01-28 16:48:10.567 29105 INFO 050_consolidated_r4 [-] Migrate pv_state 2019-01-28 16:48:10.633 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.633 29105 INFO migrate.versioning.api [-] 50 -> 51... 2019-01-28 16:48:10.645 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.645 29105 INFO migrate.versioning.api [-] 51 -> 52... 2019-01-28 16:48:10.676 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.677 29105 INFO migrate.versioning.api [-] 52 -> 53... 2019-01-28 16:48:10.770 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.770 29105 INFO migrate.versioning.api [-] 53 -> 54... 2019-01-28 16:48:10.783 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.783 29105 INFO migrate.versioning.api [-] 54 -> 55... 2019-01-28 16:48:10.852 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.852 29105 INFO migrate.versioning.api [-] 55 -> 56... 2019-01-28 16:48:10.882 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.882 29105 INFO migrate.versioning.api [-] 56 -> 57... 2019-01-28 16:48:10.893 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.894 29105 INFO migrate.versioning.api [-] 57 -> 58... 2019-01-28 16:48:10.974 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.974 29105 INFO migrate.versioning.api [-] 58 -> 59... 2019-01-28 16:48:10.986 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:10.986 29105 INFO migrate.versioning.api [-] 59 -> 60... 2019-01-28 16:48:11.017 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:11.017 29105 INFO migrate.versioning.api [-] 60 -> 61... 2019-01-28 16:48:11.040 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:11.040 29105 INFO migrate.versioning.api [-] 61 -> 62... 2019-01-28 16:48:11.057 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:11.057 29105 INFO migrate.versioning.api [-] 62 -> 63... 2019-01-28 16:48:11.820 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:11.820 29105 INFO migrate.versioning.api [-] 63 -> 64... 2019-01-28 16:48:11.830 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:11.830 29105 INFO migrate.versioning.api [-] 64 -> 65... 2019-01-28 16:48:11.904 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:11.904 29105 INFO migrate.versioning.api [-] 65 -> 66... 2019-01-28 16:48:11.948 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:11.948 29105 INFO migrate.versioning.api [-] 66 -> 67... 2019-01-28 16:48:11.982 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:11.982 29105 INFO migrate.versioning.api [-] 67 -> 68... 2019-01-28 16:48:12.033 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.033 29105 INFO migrate.versioning.api [-] 68 -> 69... 2019-01-28 16:48:12.046 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.046 29105 INFO migrate.versioning.api [-] 69 -> 70... 2019-01-28 16:48:12.050 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.050 29105 INFO migrate.versioning.api [-] 70 -> 71... 2019-01-28 16:48:12.072 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.073 29105 INFO migrate.versioning.api [-] 71 -> 72... 2019-01-28 16:48:12.073 29105 INFO 072_remove_ceilometer_service_parameter [-] Deleting ceilometer metering_time_to_live service parameter 2019-01-28 16:48:12.085 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.085 29105 INFO migrate.versioning.api [-] 72 -> 73... 2019-01-28 16:48:12.103 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.104 29105 INFO migrate.versioning.api [-] 73 -> 74... 2019-01-28 16:48:12.124 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.124 29105 INFO migrate.versioning.api [-] 74 -> 75... 2019-01-28 16:48:12.142 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.142 29105 INFO migrate.versioning.api [-] 75 -> 76... 2019-01-28 16:48:12.181 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.182 29105 INFO migrate.versioning.api [-] 76 -> 77... 2019-01-28 16:48:12.278 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.278 29105 INFO migrate.versioning.api [-] 77 -> 78... 2019-01-28 16:48:12.348 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.348 29105 INFO migrate.versioning.api [-] 78 -> 79... 2019-01-28 16:48:12.369 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.369 29105 INFO migrate.versioning.api [-] 79 -> 80... 2019-01-28 16:48:12.408 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.408 29105 INFO migrate.versioning.api [-] 80 -> 81... 2019-01-28 16:48:12.419 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.419 29105 INFO migrate.versioning.api [-] 81 -> 82... 2019-01-28 16:48:12.429 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.429 29105 INFO migrate.versioning.api [-] 82 -> 83... 2019-01-28 16:48:12.484 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:12.484 29105 INFO migrate.versioning.api [-] 83 -> 84... 2019-01-28 16:48:12.601 29105 INFO migrate.versioning.api [-] done 2019-01-28 16:48:14.665 29263 INFO sysinv.conductor.ceph [-] System is not configured. Cannot create Cluster DB entry 2019-01-28 16:48:14.836 29263 INFO sysinv.conductor.manager [-] sysinv-conductor start committed system={'system_mode': None, 'created_at': datetime.datetime(2019, 1, 28, 16, 48, 14, 669370, tzinfo=), 'uuid': u'f1ccce70-4b53-4a09-a9fb-803d187a8318', 'software_version': u'19.01', 'service_project_name': None, 'system_type': None, 'name': u'06d5c304-20d0-4d34-b76a-42036df314dc', 'description': None, 'location': None, 'updated_at': None, 'capabilities': {}, 'id': 1, 'contact': None, 'security_feature': u'nopti nospectre_v2', 'services': 72, 'timezone': None, 'security_profile': u'standard', 'distributed_cloud_role': None, 'region_name': None} 2019-01-28 16:48:14.864 29263 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on localhost:5672 2019-01-28 16:48:48.698 32485 INFO keystonemiddleware.auth_token [-] Starting Keystone auth_token middleware 2019-01-28 16:48:48.701 32485 WARNING keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. 2019-01-28 16:48:48.704 32485 INFO oslo.service.wsgi [-] sysinv_api listening on 0.0.0.0:6385 2019-01-28 16:48:48.705 32485 INFO oslo_service.service [-] Starting 2 workers 2019-01-28 16:48:59.152 32660 WARNING keystonemiddleware.auth_token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. 2019-01-28 16:48:59.896 32661 WARNING keystonemiddleware.auth_token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. 2019-01-28 16:49:00.663 32661 INFO sysinv.api.controllers.v1.system [-] update system mode simplex 2019-01-28 16:49:01.067 32661 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on localhost:5672 2019-01-28 16:49:01.095 29263 INFO sysinv.conductor.manager [req-1c3a0833-0293-4bd5-9dc3-ad54940f06fc admin admin] _config_update_hosts config_uuid=f4b79304-df15-4e49-9c96-24cedf97a432 2019-01-28 16:49:01.101 29263 INFO sysinv.openstack.common.rpc.common [req-1c3a0833-0293-4bd5-9dc3-ad54940f06fc admin admin] Connected to AMQP server on localhost:5672 2019-01-28 16:49:01.105 32661 INFO sysinv.api.controllers.v1.system [-] update system timezone to UTC 2019-01-28 16:49:01.114 29263 INFO sysinv.conductor.manager [req-1c3a0833-0293-4bd5-9dc3-ad54940f06fc admin admin] _config_update_hosts config_uuid=62213cf3-56bb-48b4-b4b9-c3607c104c70 2019-01-28 16:49:01.127 29263 INFO sysinv.conductor.manager [req-1c3a0833-0293-4bd5-9dc3-ad54940f06fc admin admin] _config_update_hosts config_uuid=7e947b4b-ae53-49dd-b334-7fe2d2168e61 2019-01-28 16:49:02.057 32661 WARNING sysinv.api.controllers.v1.dns [-] dns {'uuid': u'd9b17baa-0ac2-41cf-8524-9184aa1cf8cf', 'nameservers': u'10.248.2.1,10.19.1.4,10.2.71.6', 'created_at': datetime.datetime(2019, 1, 28, 16, 48, 14, 690469, tzinfo=), 'updated_at': None, 'action': None, 'isystem_uuid': u'f1ccce70-4b53-4a09-a9fb-803d187a8318', 'id': 1, 'forisystemid': 1} 2019-01-28 16:49:02.080 29263 INFO sysinv.conductor.manager [req-22a41b15-f691-44d9-baa7-bbde8c771a80 admin admin] _config_update_hosts config_uuid=580a423d-c5dc-4f7f-a45c-2cbee5587aff 2019-01-28 16:49:02.124 32661 INFO sysinv.api.controllers.v1.host [-] SYS_I host controller-0 ihost_post_start_2019-01-28-16-49-02 add 2019-01-28 16:49:02.140 32661 INFO sysinv.api.controllers.v1.host [-] update_subfunctions: subfunctions not set. personality=controller 2019-01-28 16:49:02.140 32661 INFO sysinv.api.controllers.v1.host [-] bm_ip in delta=set(['tboot', 'boot_device', 'console', 'uuid', 'mgmt_ip', 'subfunctions', 'bm_password', 'hostname', 'availability', 'capabilities', 'rootfs_device', 'bm_username', 'location', 'bm_ip', 'bm_type', 'invprovision', 'operational', 'administrative', 'install_output', 'mgmt_mac', 'personality']) obm_ip= nbm_ip= 2019-01-28 16:49:02.180 32661 INFO sysinv.api.controllers.v1.host [-] controller-0 bm semantic checks for user_agent Python-httplib2/0.9.2 (gzip) passed 2019-01-28 16:49:02.204 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Local Region Name: RegionOne 2019-01-28 16:49:02.211 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Disk size : 600 ... large disk defaults 2019-01-28 16:49:02.211 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:backup:backup-lv 50 2019-01-28 16:49:02.232 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:cgcs:cgcs-lv 10 2019-01-28 16:49:02.239 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:database:pgsql-lv 20 2019-01-28 16:49:02.245 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:scratch:scratch-lv 8 2019-01-28 16:49:02.251 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:gnocchi:gnocchi-lv 5 2019-01-28 16:49:02.257 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:img-conversions:img-conversions-lv 20 2019-01-28 16:49:02.263 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:extension:extension-lv 1 2019-01-28 16:49:02.269 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:docker:docker-lv 30 2019-01-28 16:49:02.275 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:etcd:etcd-lv 5 2019-01-28 16:49:02.283 29263 INFO sysinv.conductor.manager [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Creating FS:docker-distribution:dockerdistribution-lv 16 2019-01-28 16:49:02.368 29263 INFO sysinv.db.sqlalchemy.api [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] _host_upgrade_create system version=19.01 2019-01-28 16:49:02.377 29263 INFO sysinv.common.fm [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Generating FM Customer Log {'hostid': u'controller-0', 'entity_type': 'host', 'event_id': '200.020', 'entity': u'host=controller-0.event=discovered', 'fm_severity': 'clear', 'reason_text': u"controller-0 has been 'discovered'.", 'fm_event_type': 'equipment'} 2019-01-28 16:49:02.378 29263 WARNING root [-] fm_python_extension: Failed to connect to FM manager 2019-01-28 16:49:02.378 29263 ERROR sysinv.common.fm [req-27544cb3-386f-41a5-bf20-821de2e14003 admin admin] Failed to generate customer log, fm_uuid=None. 2019-01-28 16:49:04.246 23745 INFO sysinv.agent.manager [-] get_ihost_by_macs rpc Timeout. 2019-01-28 16:49:04.255 23745 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on localhost:5672 2019-01-28 16:49:04.273 23745 WARNING sysinv.openstack.common.loopingcall [-] task run outlasted interval by 1.064711 sec 2019-01-28 16:49:04.274 23745 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 16:49:04.726 23745 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 16:49:04.729 23745 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 16:49:04.730 23745 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 16:49:04.952 23745 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 16:49:04.955 23745 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 16:49:04.955 23745 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 16:49:05.178 23745 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 16:49:05.181 23745 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 16:49:05.181 23745 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 16:49:05.236 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] controller-0 updating action_state=reinstalled 2019-01-28 16:49:05.248 23745 INFO sysinv.agent.manager [-] Removed /etc/platform/.first_boot 2019-01-28 16:49:05.249 23745 INFO sysinv.agent.manager [-] Sysinv Agent platform update by host: {'action_state': 'reinstalled'} 2019-01-28 16:49:05.249 23745 INFO sysinv.agent.manager [-] Agent found matching ihost: 3a53c151-5f31-42c5-9224-b754ff15d44c 2019-01-28 16:49:05.587 29263 ERROR sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Failed in neutron stuff 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager Traceback (most recent call last): 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 4385, in subfunctions_update_by_ihost 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager context, ihost['hostname']) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/openstack.py", line 247, in get_neutron_host_id_by_name 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager hosts = client.list_hosts() 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1448, in list_hosts 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager return self.get(self.hosts_path, params=_params) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 355, in get 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 332, in retry_request 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 283, in do_request 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager resp, replybody = self.httpclient.do_request(action, method, body=body) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 343, in do_request 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager self._check_uri_length(url) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 336, in _check_uri_length 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager uri_len = len(self.endpoint_url) + len(url) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 350, in endpoint_url 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager return self.get_endpoint() 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 226, in get_endpoint 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager return self.session.get_endpoint(auth or self.auth, **kwargs) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 705, in get_endpoint 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager return auth.get_endpoint(self, **kwargs) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 332, in get_endpoint 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager service_name=service_name) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager return wrapped(*args, **kwargs) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/service_catalog.py", line 297, in url_for 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager raise exceptions.EndpointNotFound(msg) 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager EndpointNotFound: internalURL endpoint for network service in RegionOne region not found 2019-01-28 16:49:05.587 29263 TRACE sysinv.conductor.manager 2019-01-28 16:49:05.702 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new port {u'dev_id': 0, u'sriov_numvfs': 0, u'link_mode': u'0', u'driver': u'e1000', 'bootp': None, u'prevision': u'-r03', u'dpdksupport': False, u'mac': u'52:54:00:4c:4d:a5', u'pname': u'ens3', 'host_id': 1, 'interface_id': 1, u'speed': u'1000', u'psvendor': u'Red Hat, Inc.', u'numa_node': u'-1', u'pclass': u'Ethernet controller', u'mtu': u'1500', u'pvendor': u'Intel Corporation', u'sriov_vfs_pci_address': u'', u'psdevice': u'QEMU Virtual Machine', u'pciaddr': u'0000:00:03.0', u'pdevice': u'82540EM Gigabit Ethernet Controller', u'sriov_totalvfs': None} on host 3a53c151-5f31-42c5-9224-b754ff15d44c 2019-01-28 16:49:05.778 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new port {u'dev_id': 0, u'sriov_numvfs': 0, u'link_mode': u'0', u'driver': u'e1000', 'bootp': None, u'prevision': u'-r03', u'dpdksupport': False, u'mac': u'52:54:00:9a:ae:34', u'pname': u'ens4', 'host_id': 1, 'interface_id': 2, u'speed': None, u'psvendor': u'Red Hat, Inc.', u'numa_node': u'-1', u'pclass': u'Ethernet controller', u'mtu': u'1500', u'pvendor': u'Intel Corporation', u'sriov_vfs_pci_address': u'', u'psdevice': u'QEMU Virtual Machine', u'pciaddr': u'0000:00:04.0', u'pdevice': u'82540EM Gigabit Ethernet Controller', u'sriov_totalvfs': None} on host 3a53c151-5f31-42c5-9224-b754ff15d44c 2019-01-28 16:49:05.858 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new port {u'dev_id': 0, u'sriov_numvfs': 0, u'link_mode': u'0', u'driver': u'virtio-pci', 'bootp': None, u'prevision': u'0', u'dpdksupport': True, u'mac': u'52:54:00:3b:b7:9c', u'pname': u'eth1000', 'host_id': 1, 'interface_id': 3, u'speed': None, u'psvendor': u'Red Hat, Inc.', u'numa_node': u'-1', u'pclass': u'Ethernet controller', u'mtu': u'1500', u'pvendor': u'Red Hat, Inc.', u'sriov_vfs_pci_address': u'', u'psdevice': u'Device 0001', u'pciaddr': u'0000:00:05.0', u'pdevice': u'Virtio network device', u'sriov_totalvfs': None} on host 3a53c151-5f31-42c5-9224-b754ff15d44c 2019-01-28 16:49:05.942 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new port {u'dev_id': 0, u'sriov_numvfs': 0, u'link_mode': u'0', u'driver': u'virtio-pci', 'bootp': None, u'prevision': u'0', u'dpdksupport': True, u'mac': u'52:54:00:e2:a7:a4', u'pname': u'eth1001', 'host_id': 1, 'interface_id': 4, u'speed': None, u'psvendor': u'Red Hat, Inc.', u'numa_node': u'-1', u'pclass': u'Ethernet controller', u'mtu': u'1500', u'pvendor': u'Red Hat, Inc.', u'sriov_vfs_pci_address': u'', u'psdevice': u'Device 0001', u'pciaddr': u'0000:00:06.0', u'pdevice': u'Virtio network device', u'sriov_totalvfs': None} on host 3a53c151-5f31-42c5-9224-b754ff15d44c 2019-01-28 16:49:05.974 29263 ERROR sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Failed in neutron stuff 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager Traceback (most recent call last): 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 4385, in subfunctions_update_by_ihost 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager context, ihost['hostname']) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/openstack.py", line 247, in get_neutron_host_id_by_name 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager hosts = client.list_hosts() 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1448, in list_hosts 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager return self.get(self.hosts_path, params=_params) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 355, in get 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 332, in retry_request 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 283, in do_request 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager resp, replybody = self.httpclient.do_request(action, method, body=body) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 343, in do_request 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager self._check_uri_length(url) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 336, in _check_uri_length 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager uri_len = len(self.endpoint_url) + len(url) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 350, in endpoint_url 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager return self.get_endpoint() 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 226, in get_endpoint 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager return self.session.get_endpoint(auth or self.auth, **kwargs) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 705, in get_endpoint 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager return auth.get_endpoint(self, **kwargs) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 332, in get_endpoint 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager service_name=service_name) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager return wrapped(*args, **kwargs) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/service_catalog.py", line 297, in url_for 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager raise exceptions.EndpointNotFound(msg) 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager EndpointNotFound: internalURL endpoint for network service in RegionOne region not found 2019-01-28 16:49:05.974 29263 TRACE sysinv.conductor.manager 2019-01-28 16:49:06.006 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new device {u'sriov_numvfs': 0, u'driver': u'ata_piix', u'prevision': u'-p80', 'host_id': 1, u'psvendor': u'Red Hat, Inc.', u'extra_info': None, u'name': u'pci_0000_00_01_1', u'numa_node': u'-1', u'pdevice_id': u'7010', u'pclass': u'IDE interface', u'pvendor': u'Intel Corporation', u'sriov_vfs_pci_address': u'', u'psdevice': u'Qemu virtual machine', u'pciaddr': u'0000:00:01.1', u'pdevice': u'82371SB PIIX3 IDE [Natoma/Triton II]', u'pvendor_id': u'8086', u'pclass_id': u'010180', u'enabled': False, u'sriov_totalvfs': None} on host 1 2019-01-28 16:49:06.019 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new device {u'sriov_numvfs': 0, u'driver': u'uhci_hcd', u'prevision': u'-r01', 'host_id': 1, u'psvendor': u'Red Hat, Inc.', u'extra_info': None, u'name': u'pci_0000_00_01_2', u'numa_node': u'-1', u'pdevice_id': u'7020', u'pclass': u'USB controller', u'pvendor': u'Intel Corporation', u'sriov_vfs_pci_address': u'', u'psdevice': u'QEMU Virtual Machine', u'pciaddr': u'0000:00:01.2', u'pdevice': u'82371SB PIIX3 USB [Natoma/Triton II]', u'pvendor_id': u'8086', u'pclass_id': u'0c0300', u'enabled': False, u'sriov_totalvfs': None} on host 1 2019-01-28 16:49:06.032 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new device {u'sriov_numvfs': 0, u'driver': None, u'prevision': u'Red Hat, Inc.', 'host_id': 1, u'psvendor': u'QEMU Virtual Machine', u'extra_info': None, u'name': u'pci_0000_00_02_0', u'numa_node': u'-1', u'pdevice_id': u'00b8', u'pclass': u'VGA compatible controller', u'pvendor': u'Cirrus Logic', u'sriov_vfs_pci_address': u'', u'psdevice': u'QEMU Virtual Machine', u'pciaddr': u'0000:00:02.0', u'pdevice': u'GD 5446', u'pvendor_id': u'1013', u'pclass_id': u'030000', u'enabled': True, u'sriov_totalvfs': None} on host 1 2019-01-28 16:49:06.046 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new device {u'sriov_numvfs': 0, u'driver': u'ahci', u'prevision': u'-r02', 'host_id': 1, u'psvendor': u'-p01', u'extra_info': None, u'name': u'pci_0000_00_07_0', u'numa_node': u'-1', u'pdevice_id': u'2922', u'pclass': u'SATA controller', u'pvendor': u'Intel Corporation', u'sriov_vfs_pci_address': u'', u'psdevice': u'Red Hat, Inc.', u'pciaddr': u'0000:00:07.0', u'pdevice': u'82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]', u'pvendor_id': u'8086', u'pclass_id': u'010601', u'enabled': False, u'sriov_totalvfs': None} on host 1 2019-01-28 16:49:06.059 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Attempting to create new device {u'sriov_numvfs': 0, u'driver': u'virtio-pci', u'prevision': u'0', 'host_id': 1, u'psvendor': u'Red Hat, Inc.', u'extra_info': None, u'name': u'pci_0000_00_08_0', u'numa_node': u'-1', u'pdevice_id': u'1002', u'pclass': u'Unclassified device [00ff]', u'pvendor': u'Red Hat, Inc.', u'sriov_vfs_pci_address': u'', u'psdevice': u'Device 0005', u'pciaddr': u'0000:00:08.0', u'pdevice': u'Virtio memory balloon', u'pvendor_id': u'1af4', u'pclass_id': u'00ff00', u'enabled': False, u'sriov_totalvfs': None} on host 1 2019-01-28 16:49:06.218 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Logical CPU topology: host:controller-0 (controller,worker), sockets:1, cores/socket=6, threads/core=1, reference:current (initial) 2019-01-28 16:49:06.218 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] cpu_id : 0 1 2 3 4 5 2019-01-28 16:49:06.218 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] socket_id : 0 0 0 0 0 0 2019-01-28 16:49:06.218 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] core_id : 0 1 2 3 4 5 2019-01-28 16:49:06.219 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] thread_id : 0 0 0 0 0 0 2019-01-28 16:49:08.568 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Partition create on host: 1. Details: {'status': 1, 'forihostid': 1, u'device_path': u'/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part1', u'start_mib': u'1', u'uuid': u'491211cd-f682-41ca-bc32-67f019b7b95b', 'idisk_id': 1, u'type_name': u'BIOS boot partition', u'device_node': u'/dev/sda1', u'end_mib': u'2', 'idisk_uuid': u'18101309-ae81-4916-867f-d33f45943f28', u'type_guid': u'21686148-6449-6e6f-744e-656564454649', u'size_mib': u'1'} 2019-01-28 16:49:08.670 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Partition create on host: 1. Details: {'status': 1, 'forihostid': 1, u'device_path': u'/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part2', u'start_mib': u'2', u'uuid': u'24932520-86e1-4917-a18f-f5b26d88040c', 'idisk_id': 1, u'type_name': u'Microsoft basic data', u'device_node': u'/dev/sda2', u'end_mib': u'502', 'idisk_uuid': u'18101309-ae81-4916-867f-d33f45943f28', u'type_guid': u'ebd0a0a2-b9e5-4433-87c0-68b6b72699c7', u'size_mib': u'500'} 2019-01-28 16:49:08.687 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Partition create on host: 1. Details: {'status': 1, 'forihostid': 1, u'device_path': u'/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part3', u'start_mib': u'502', u'uuid': u'e88cae09-e541-4edb-8bed-235d73800070', 'idisk_id': 1, u'type_name': u'Microsoft basic data', u'device_node': u'/dev/sda3', u'end_mib': u'20502', 'idisk_uuid': u'18101309-ae81-4916-867f-d33f45943f28', u'type_guid': u'ebd0a0a2-b9e5-4433-87c0-68b6b72699c7', u'size_mib': u'20000'} 2019-01-28 16:49:08.703 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Partition create on host: 1. Details: {'status': 1, 'forihostid': 1, u'device_path': u'/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part4', u'start_mib': u'20502', u'uuid': u'40be4c0d-7110-434b-819d-cbb92db19b04', 'idisk_id': 1, u'type_name': u'Linux LVM', u'device_node': u'/dev/sda4', u'end_mib': u'244758', 'idisk_uuid': u'18101309-ae81-4916-867f-d33f45943f28', u'type_guid': u'e6d6d379-f507-44c2-a23c-238f2a3df928', u'size_mib': u'224256'} 2019-01-28 16:49:08.789 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] Inconsistent Data: Not adding PV: {u'lvm_pe_alloced': 788, 'forihostid': 1, u'lvm_pe_total': 7007, u'lvm_vg_name': u'cgts-vg', u'lvm_pv_name': u'/dev/sda4', u'lvm_pv_uuid': u'HOPuNL-eohy-2db4-bZnm-kl9q-2d85-QhHjoG', u'lvm_pv_size': 235115905024} 2019-01-28 16:49:08.886 29263 INFO sysinv.conductor.manager [req-0c398a4a-f041-4346-b260-416bdb0ddab0 None None] platform_interfaces host_id=1 info_list=[] 2019-01-28 16:49:10.544 23745 INFO sysinv.agent.manager [req-22a41b15-f691-44d9-baa7-bbde8c771a80 admin admin] Agent config applied 580a423d-c5dc-4f7f-a45c-2cbee5587aff 2019-01-28 16:49:10.563 29263 WARNING sysinv.conductor.manager [req-22a41b15-f691-44d9-baa7-bbde8c771a80 admin admin] controller-0: iconfig no target, but config 580a423d-c5dc-4f7f-a45c-2cbee5587aff applied 2019-01-28 16:49:10.563 29263 INFO sysinv.conductor.manager [req-22a41b15-f691-44d9-baa7-bbde8c771a80 admin admin] SYS_I Clear system config alarm: controller-0 2019-01-28 16:49:10.564 29263 WARNING root [-] fm_python_extension: Failed to connect to FM manager 2019-01-28 16:49:26.080 32661 INFO sysinv.api.controllers.v1.interface [-] Setting MAC of interface lo to 00:00:00:00:00:00; taken from None 2019-01-28 16:49:26.109 32661 INFO sysinv.api.controllers.v1.interface [-] _update_interface_datanetworks interface= datanetworks=None pns=[] 2019-01-28 16:53:02.678 23745 ERROR sysinv.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common Traceback (most recent call last): 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 564, in ensure 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common return method(*args, **kwargs) 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 644, in _consume 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common return connection.drain_events(**kwargs) 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 464, in drain_events 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common return self.blocking_read(timeout) 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 468, in blocking_read 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common frame = self.transport.read_frame() 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 237, in read_frame 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common frame_header = read(7, True) 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 385, in _read 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common raise IOError('Socket closed') 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common IOError: Socket closed 2019-01-28 16:53:02.678 23745 TRACE sysinv.openstack.common.rpc.common 2019-01-28 16:53:02.680 23745 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on localhost:5672 2019-01-28 16:53:02.703 23745 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on localhost:5672 2019-01-28 16:53:04.310 23745 ERROR sysinv.openstack.common.rpc.common [-] Failed to declare consumer for topic 'b1d1633d0fbf4dd7b8a306ade4ddec60': Socket closed 2019-01-28 16:53:04.310 23745 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on localhost:5672 2019-01-28 16:53:04.326 23745 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on localhost:5672 2019-01-28 16:53:04.983 29263 ERROR sysinv.openstack.common.rpc.common [-] Failed to consume message from queue: Socket closed 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common Traceback (most recent call last): 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 564, in ensure 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common return method(*args, **kwargs) 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 644, in _consume 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common return connection.drain_events(**kwargs) 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 464, in drain_events 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common return self.blocking_read(timeout) 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 468, in blocking_read 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common frame = self.transport.read_frame() 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 237, in read_frame 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common frame_header = read(7, True) 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 385, in _read 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common raise IOError('Socket closed') 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common IOError: Socket closed 2019-01-28 16:53:04.983 29263 TRACE sysinv.openstack.common.rpc.common 2019-01-28 16:53:04.984 29263 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on localhost:5672 2019-01-28 16:53:05.001 29263 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on localhost:5672 2019-01-28 16:53:05.013 29263 ERROR sysinv.openstack.common.rpc.common [req-2e010587-7293-45a7-8ce5-97a37c2874b6 None None] Failed to publish message to topic 'b1d1633d0fbf4dd7b8a306ade4ddec60': [Errno 104] Connection reset by peer 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common Traceback (most recent call last): 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 564, in ensure 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common return method(*args, **kwargs) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 676, in _publish 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common publisher.send(msg, timeout) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 316, in send 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common self.producer.publish(msg) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 181, in publish 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common exchange_name, declare, 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 194, in _publish 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common [maybe_declare(entity) for entity in declare] 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 102, in maybe_declare 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common return maybe_declare(entity, self.channel, retry, **retry_policy) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/common.py", line 125, in maybe_declare 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common return _maybe_declare(entity, declared, ident, channel, orig) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/common.py", line 131, in _maybe_declare 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common entity.declare(channel=channel) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 185, in declare 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common nowait=nowait, passive=passive, 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 630, in exchange_declare 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common wait=None if nowait else spec.Exchange.DeclareOk, 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 64, in send_method 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common conn.frame_writer(1, self.channel_id, sig, args, content) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 174, in write_frame 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common write(view[:offset]) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 269, in write 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common self._write(s) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 385, in sendall 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common tail = self.send(data, flags) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 379, in send 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common return self._send_loop(self.fd.send, data, flags) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 366, in _send_loop 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common return send_method(data, *args) 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common error: [Errno 104] Connection reset by peer 2019-01-28 16:53:05.013 29263 TRACE sysinv.openstack.common.rpc.common 2019-01-28 16:53:05.015 29263 INFO sysinv.openstack.common.rpc.common [req-2e010587-7293-45a7-8ce5-97a37c2874b6 None None] Reconnecting to AMQP server on localhost:5672 2019-01-28 16:53:05.021 29263 INFO sysinv.openstack.common.rpc.common [req-2e010587-7293-45a7-8ce5-97a37c2874b6 None None] Connected to AMQP server on localhost:5672 2019-01-28 16:53:12.550 14196 INFO manage-partitions [-] Called partition 'check' with 'None' 'None' and '[{"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part1", "start_mib": 1, "uuid": "491211cd-f682-41ca-bc32-67f019b7b95b", "device_node": "/dev/sda1", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "21686148-6449-6e6f-744e-656564454649", "size_mib": 1}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part2", "start_mib": 2, "uuid": "24932520-86e1-4917-a18f-f5b26d88040c", "device_node": "/dev/sda2", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 500}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part3", "start_mib": 502, "uuid": "e88cae09-e541-4edb-8bed-235d73800070", "device_node": "/dev/sda3", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 20000}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part4", "start_mib": 20502, "uuid": "40be4c0d-7110-434b-819d-cbb92db19b04", "device_node": "/dev/sda4", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "e6d6d379-f507-44c2-a23c-238f2a3df928", "size_mib": 224256}]' 2019-01-28 16:53:12.551 14196 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 16:53:12.554 14196 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 16:53:12.772 14196 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 16:53:12.782 14196 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 16:53:12.996 14196 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 16:53:13.000 14196 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 16:53:13.227 14196 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 16:53:13.239 14196 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 16:53:13.497 14196 INFO manage-partitions [-] Executing command: 'sgdisk -i 1 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 16:53:13.513 14196 INFO manage-partitions [-] Executing command: 'sgdisk -i 2 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 16:53:13.530 14196 INFO manage-partitions [-] Executing command: 'sgdisk -i 3 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 16:53:13.550 14196 INFO manage-partitions [-] Executing command: 'sgdisk -i 4 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 16:54:55.839 23745 INFO sysinv.openstack.common.service [-] Caught SIGTERM, exiting 2019-01-28 16:54:56.397 32701 INFO sysinv.agent.lldp.manager [-] Configured sysinv LLDP agent drivers: ['lldpd', 'ovs'] 2019-01-28 16:54:56.431 32701 INFO sysinv.agent.lldp.manager [-] Loaded sysinv LLDP agent drivers: ['lldpd', 'ovs'] 2019-01-28 16:54:56.432 32701 INFO sysinv.agent.lldp.manager [-] Registered sysinv LLDP agent drivers: ['lldpd', 'ovs'] 2019-01-28 16:54:56.473 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 16:54:56.477 32701 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 16:54:56.617 29263 INFO sysinv.openstack.common.service [-] Caught SIGTERM, exiting 2019-01-28 16:54:56.962 32701 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 16:54:56.965 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 16:54:56.965 32701 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 16:54:57.193 32701 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 16:54:57.196 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 16:54:57.196 32701 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 16:54:57.428 32701 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 16:54:57.431 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 16:54:57.432 32701 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 16:54:57.453 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 16:55:57.472 32701 INFO sysinv.agent.manager [-] get_ihost_by_macs rpc Timeout. 2019-01-28 16:55:57.498 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 16:56:57.523 32701 INFO sysinv.agent.manager [-] ilvg_get_nova_ilvg_by_ihost() Timeout. 2019-01-28 16:56:57.524 32701 WARNING sysinv.openstack.common.loopingcall [-] task run outlasted interval by 61.046956 sec 2019-01-28 16:56:57.526 32701 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 16:56:58.047 32701 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 16:56:58.051 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 16:56:58.051 32701 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 16:56:58.286 32701 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 16:56:58.289 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 16:56:58.289 32701 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 16:56:58.524 32701 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 16:56:58.527 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 16:56:58.528 32701 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 16:56:58.550 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 16:57:58.585 32701 INFO sysinv.agent.manager [-] get_ihost_by_macs rpc Timeout. 2019-01-28 16:57:58.594 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 16:58:58.669 32701 INFO sysinv.agent.manager [-] ilvg_get_nova_ilvg_by_ihost() Timeout. 2019-01-28 16:58:58.670 32701 WARNING sysinv.openstack.common.loopingcall [-] task run outlasted interval by 61.145099 sec 2019-01-28 16:58:58.673 32701 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 16:58:59.215 32701 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 16:58:59.218 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 16:58:59.218 32701 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 16:58:59.522 32701 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 16:58:59.526 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 16:58:59.526 32701 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 16:58:59.781 32701 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 16:58:59.784 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 16:58:59.785 32701 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 16:58:59.809 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 16:59:59.878 32701 INFO sysinv.agent.manager [-] get_ihost_by_macs rpc Timeout. 2019-01-28 16:59:59.901 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:00:59.962 32701 INFO sysinv.agent.manager [-] ilvg_get_nova_ilvg_by_ihost() Timeout. 2019-01-28 17:00:59.962 32701 WARNING sysinv.openstack.common.loopingcall [-] task run outlasted interval by 61.290553 sec 2019-01-28 17:00:59.962 32701 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 17:01:00.425 32701 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 17:01:00.429 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 17:01:00.429 32701 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 17:01:00.673 32701 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 17:01:00.677 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 17:01:00.677 32701 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 17:01:00.933 32701 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 17:01:00.937 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 17:01:00.937 32701 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 17:01:00.964 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:02:01.022 32701 INFO sysinv.agent.manager [-] get_ihost_by_macs rpc Timeout. 2019-01-28 17:02:01.028 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:03:01.089 32701 INFO sysinv.agent.manager [-] ilvg_get_nova_ilvg_by_ihost() Timeout. 2019-01-28 17:03:01.089 32701 WARNING sysinv.openstack.common.loopingcall [-] task run outlasted interval by 61.126665 sec 2019-01-28 17:03:01.089 32701 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 17:03:01.564 32701 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 17:03:01.568 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 17:03:01.568 32701 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 17:03:01.809 32701 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 17:03:01.813 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 17:03:01.813 32701 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 17:03:02.053 32701 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 17:03:02.057 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 17:03:02.057 32701 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 17:03:02.081 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:04:02.140 32701 INFO sysinv.agent.manager [-] get_ihost_by_macs rpc Timeout. 2019-01-28 17:04:02.146 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:02.205 32701 INFO sysinv.agent.manager [-] ilvg_get_nova_ilvg_by_ihost() Timeout. 2019-01-28 17:05:02.205 32701 WARNING sysinv.openstack.common.loopingcall [-] task run outlasted interval by 61.116105 sec 2019-01-28 17:05:02.206 32701 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 17:05:02.715 32701 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 17:05:02.718 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 17:05:02.719 32701 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 17:05:02.980 32701 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 17:05:02.983 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 17:05:02.984 32701 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 17:05:03.225 32701 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 17:05:03.228 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 17:05:03.229 32701 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 17:05:03.255 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:07.629 32485 INFO oslo_service.service [-] Caught SIGTERM, stopping children 2019-01-28 17:05:07.630 32485 INFO oslo.service.wsgi [-] Stopping WSGI server. 2019-01-28 17:05:07.630 32485 INFO oslo_service.service [-] Waiting on 2 children to exit 2019-01-28 17:05:07.630 32661 INFO oslo.service.wsgi [-] Stopping WSGI server. 2019-01-28 17:05:07.630 32660 INFO oslo.service.wsgi [-] Stopping WSGI server. 2019-01-28 17:05:07.637 32485 INFO oslo_service.service [-] Child 32660 exited with status 0 2019-01-28 17:05:07.637 32485 INFO oslo_service.service [-] Child 32661 exited with status 0 2019-01-28 17:05:11.607 32701 ERROR sysinv.openstack.common.rpc.common [-] Failed to consume message from queue: [Errno 104] Connection reset by peer 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common Traceback (most recent call last): 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 564, in ensure 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common return method(*args, **kwargs) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 644, in _consume 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common return connection.drain_events(**kwargs) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 464, in drain_events 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common return self.blocking_read(timeout) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 469, in blocking_read 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common return self.on_inbound_frame(frame) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 68, in on_frame 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common callback(channel, method_sig, buf, None) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 473, in on_inbound_method 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common method_sig, payload, content, 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 142, in dispatch_method 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common listener(*args) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 593, in _on_close 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common self._x_close_ok() 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 608, in _x_close_ok 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common self.send_method(spec.Connection.CloseOk, callback=self._on_close_ok) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 64, in send_method 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common conn.frame_writer(1, self.channel_id, sig, args, content) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 174, in write_frame 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common write(view[:offset]) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 269, in write 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common self._write(s) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 385, in sendall 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common tail = self.send(data, flags) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 379, in send 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common return self._send_loop(self.fd.send, data, flags) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 366, in _send_loop 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common return send_method(data, *args) 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common error: [Errno 104] Connection reset by peer 2019-01-28 17:05:11.607 32701 TRACE sysinv.openstack.common.rpc.common 2019-01-28 17:05:11.608 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:11.613 32701 ERROR sysinv.openstack.common.rpc.common [-] Failed to consume message from queue: [Errno 104] Connection reset by peer 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common Traceback (most recent call last): 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 564, in ensure 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common return method(*args, **kwargs) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 644, in _consume 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common return connection.drain_events(**kwargs) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 464, in drain_events 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common return self.blocking_read(timeout) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 469, in blocking_read 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common return self.on_inbound_frame(frame) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 68, in on_frame 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common callback(channel, method_sig, buf, None) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 473, in on_inbound_method 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common method_sig, payload, content, 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 142, in dispatch_method 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common listener(*args) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 593, in _on_close 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common self._x_close_ok() 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 608, in _x_close_ok 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common self.send_method(spec.Connection.CloseOk, callback=self._on_close_ok) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 64, in send_method 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common conn.frame_writer(1, self.channel_id, sig, args, content) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 174, in write_frame 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common write(view[:offset]) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 269, in write 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common self._write(s) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 385, in sendall 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common tail = self.send(data, flags) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 379, in send 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common return self._send_loop(self.fd.send, data, flags) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 366, in _send_loop 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common return send_method(data, *args) 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common error: [Errno 104] Connection reset by peer 2019-01-28 17:05:11.613 32701 TRACE sysinv.openstack.common.rpc.common 2019-01-28 17:05:11.614 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:11.618 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2019-01-28 17:05:11.618 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2019-01-28 17:05:12.620 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:12.636 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:12.651 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds. 2019-01-28 17:05:12.651 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds. 2019-01-28 17:05:15.652 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:15.656 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds. 2019-01-28 17:05:15.656 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:15.660 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds. 2019-01-28 17:05:20.657 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:20.662 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 7 seconds. 2019-01-28 17:05:20.662 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:20.666 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 7 seconds. 2019-01-28 17:05:27.664 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:27.668 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 9 seconds. 2019-01-28 17:05:27.668 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:27.672 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 9 seconds. 2019-01-28 17:05:36.669 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:36.673 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 11 seconds. 2019-01-28 17:05:36.673 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:36.677 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 11 seconds. 2019-01-28 17:05:47.674 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:47.678 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 13 seconds. 2019-01-28 17:05:47.678 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:05:47.682 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 13 seconds. 2019-01-28 17:06:00.681 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:06:00.701 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 15 seconds. 2019-01-28 17:06:00.702 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:06:00.715 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 15 seconds. 2019-01-28 17:06:10.351 20819 INFO keystonemiddleware.auth_token [-] Starting Keystone auth_token middleware 2019-01-28 17:06:10.353 20819 WARNING keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. 2019-01-28 17:06:10.357 20819 INFO oslo.service.wsgi [-] sysinv_api listening on 192.168.204.2:6385 2019-01-28 17:06:10.357 20819 INFO oslo_service.service [-] Starting 1 workers 2019-01-28 17:06:10.362 20819 INFO keystonemiddleware.auth_token [-] Starting Keystone auth_token middleware 2019-01-28 17:06:10.365 20819 WARNING keystonemiddleware._common.config [-] The option "user_domain_name" is not known to keystonemiddleware 2019-01-28 17:06:10.365 20819 WARNING keystonemiddleware._common.config [-] The option "auth_url" is not known to keystonemiddleware 2019-01-28 17:06:10.365 20819 WARNING keystonemiddleware._common.config [-] The option "project_domain_name" is not known to keystonemiddleware 2019-01-28 17:06:10.365 20819 WARNING keystonemiddleware._common.config [-] The option "username" is not known to keystonemiddleware 2019-01-28 17:06:10.365 20819 WARNING keystonemiddleware._common.config [-] The option "project_name" is not known to keystonemiddleware 2019-01-28 17:06:10.366 20819 WARNING keystonemiddleware._common.config [-] The option "password" is not known to keystonemiddleware 2019-01-28 17:06:10.366 20819 WARNING keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. 2019-01-28 17:06:10.369 20819 INFO oslo.service.wsgi [-] sysinv_api_pxe listening on 169.254.202.2:6385 2019-01-28 17:06:10.369 20819 INFO oslo_service.service [-] Starting 1 workers 2019-01-28 17:06:10.421 20749 WARNING sysinv.conductor.ceph [-] ceph_api.fsid failed: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0.1/fsid (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] ECONNREFUSED',)) 2019-01-28 17:06:10.422 20749 INFO sysinv.conductor.ceph [-] Create new cluster record: fsid=None. 2019-01-28 17:06:10.461 20749 INFO sysinv.conductor.manager [-] sysinv-conductor start committed system={'system_mode': u'simplex', 'created_at': datetime.datetime(2019, 1, 28, 16, 48, 14, 669370, tzinfo=), 'uuid': u'f1ccce70-4b53-4a09-a9fb-803d187a8318', 'software_version': u'19.01', 'service_project_name': u'services', 'system_type': u'All-in-one', 'name': u'06d5c304-20d0-4d34-b76a-42036df314dc', 'description': None, 'location': None, 'updated_at': datetime.datetime(2019, 1, 28, 16, 49, 0, 660043, tzinfo=), 'capabilities': {u'shared_services': u'[]', u'sdn_enabled': False, u'vswitch_type': u'ovs-dpdk', u'https_enabled': False, u'kubernetes_enabled': True, u'region_config': False}, 'id': 1, 'contact': None, 'security_feature': u'nopti nospectre_v2', 'services': 72, 'timezone': u'UTC', 'security_profile': u'standard', 'distributed_cloud_role': None, 'region_name': u'RegionOne'} 2019-01-28 17:06:10.498 20749 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:06:10.677 21006 WARNING keystonemiddleware.auth_token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. 2019-01-28 17:06:11.778 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:06:14.489 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-06-14 patch 2019-01-28 17:06:14.489 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['action'] 2019-01-28 17:06:14.490 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 Pending update_vim_progress_status services-disabled 2019-01-28 17:06:14.490 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 action=services-disabled ihost_val_prenotify: {} ihost_val: {} 2019-01-28 17:06:14.490 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 action_check action=services-disabled, notify_vim=False notify_mtce=True rc=True 2019-01-28 17:06:14.490 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 post action_check hostupdate action=services-disabled notify_vim=False notify_mtc=True skip_notify_mtce=False 2019-01-28 17:06:14.490 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 stage_action services-disabled 2019-01-28 17:06:14.490 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 _handle_vim_services_disabled' 2019-01-28 17:06:14.490 21006 WARNING sysinv.api.controllers.v1.host [-] controller-0 Admin action task not Locking or Force Locking upon receipt of 'services-disabled'. 2019-01-28 17:06:14.490 21006 WARNING sysinv.api.controllers.v1.host [-] controller-0 Exit _handle_vim_services_disabled 2019-01-28 17:06:14.491 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 stage_action aborting request services-disabled ['action'] 2019-01-28 17:06:14.491 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_end stage_action rc exit_return_host 2019-01-28 17:06:15.710 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:06:15.715 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:06:15.722 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:06:15.727 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:06:18.640 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-06-18 patch 2019-01-28 17:06:18.640 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['availability'] 2019-01-28 17:06:18.664 21006 INFO sysinv.api.controllers.v1.host [-] host controller-0 ihost_patch_end_2019-01-28-17-06-18 patch 2019-01-28 17:06:18.700 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-06-18 patch 2019-01-28 17:06:18.701 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['subfunction_avail'] 2019-01-28 17:06:18.719 21006 INFO sysinv.api.controllers.v1.host [-] host controller-0 ihost_patch_end_2019-01-28-17-06-18 patch 2019-01-28 17:07:15.787 32701 INFO sysinv.agent.manager [-] get_ihost_by_macs rpc Timeout. 2019-01-28 17:07:15.808 32701 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:07:15.829 20749 INFO sysinv.openstack.common.rpc.common [req-c19b3cf0-7216-4647-a359-0a1c9f3107e4 None None] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:07:15.834 32701 WARNING sysinv.openstack.common.loopingcall [-] task run outlasted interval by 73.628147 sec 2019-01-28 17:07:15.834 32701 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 17:07:16.304 32701 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 17:07:16.307 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 17:07:16.307 32701 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 17:07:16.548 32701 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 17:07:16.552 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 17:07:16.552 32701 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 17:07:16.791 32701 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 17:07:16.794 32701 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 17:07:16.795 32701 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 17:07:16.857 32701 INFO sysinv.agent.manager [-] Sysinv Agent platform update by host: {} 2019-01-28 17:07:16.858 32701 INFO sysinv.agent.manager [-] Agent found matching ihost: 3a53c151-5f31-42c5-9224-b754ff15d44c 2019-01-28 17:07:17.239 20749 ERROR sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] Failed in neutron stuff 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager Traceback (most recent call last): 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 4385, in subfunctions_update_by_ihost 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager context, ihost['hostname']) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/openstack.py", line 247, in get_neutron_host_id_by_name 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager hosts = client.list_hosts() 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1448, in list_hosts 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager return self.get(self.hosts_path, params=_params) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 355, in get 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 332, in retry_request 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 283, in do_request 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager resp, replybody = self.httpclient.do_request(action, method, body=body) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 344, in do_request 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager return self.request(url, method, **kwargs) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 332, in request 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager resp = super(SessionClient, self).request(*args, **kwargs) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 192, in request 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager return self.session.request(url, method, **kwargs) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager return wrapped(*args, **kwargs) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 430, in request 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager resp = send(**kwargs) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 474, in _send_request 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager raise exceptions.ConnectionRefused(msg) 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager ConnectFailure: Unable to establish connection to http://192.168.204.2:9696/v2.0/hosts 2019-01-28 17:07:17.239 20749 TRACE sysinv.conductor.manager 2019-01-28 17:07:17.472 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] port 6f08deda-c34f-40d6-9c24-f39a49e62236 update attr: {'sriov_numvfs': 0, 'driver': u'e1000', 'sriov_vfs_pci_address': u'', 'sriov_totalvfs': None, 'dpdksupport': False, 'speed': u'1000'} 2019-01-28 17:07:17.551 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] port c6876c8f-09d6-4068-960a-d7bd546de3b2 update attr: {'sriov_numvfs': 0, 'driver': u'e1000', 'sriov_vfs_pci_address': u'', 'sriov_totalvfs': None, 'dpdksupport': False, 'speed': None} 2019-01-28 17:07:17.631 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] port 2e944997-9b41-4d13-b3e6-2670d373d1e1 update attr: {'sriov_numvfs': 0, 'driver': u'virtio-pci', 'sriov_vfs_pci_address': u'', 'sriov_totalvfs': None, 'dpdksupport': True, 'speed': None} 2019-01-28 17:07:17.709 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] port 0b435963-e0f6-47cc-a642-cddae4657b74 update attr: {'sriov_numvfs': 0, 'driver': u'virtio-pci', 'sriov_vfs_pci_address': u'', 'sriov_totalvfs': None, 'dpdksupport': True, 'speed': None} 2019-01-28 17:07:17.746 20749 ERROR sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] Failed in neutron stuff 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager Traceback (most recent call last): 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 4385, in subfunctions_update_by_ihost 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager context, ihost['hostname']) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/openstack.py", line 247, in get_neutron_host_id_by_name 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager hosts = client.list_hosts() 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1448, in list_hosts 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager return self.get(self.hosts_path, params=_params) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 355, in get 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 332, in retry_request 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 283, in do_request 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager resp, replybody = self.httpclient.do_request(action, method, body=body) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 344, in do_request 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager return self.request(url, method, **kwargs) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 332, in request 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager resp = super(SessionClient, self).request(*args, **kwargs) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 192, in request 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager return self.session.request(url, method, **kwargs) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager return wrapped(*args, **kwargs) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 430, in request 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager resp = send(**kwargs) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 474, in _send_request 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager raise exceptions.ConnectionRefused(msg) 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager ConnectFailure: Unable to establish connection to http://192.168.204.2:9696/v2.0/hosts 2019-01-28 17:07:17.746 20749 TRACE sysinv.conductor.manager 2019-01-28 17:07:17.781 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] attr: {'sriov_numvfs': 0, 'driver': u'ata_piix', 'psvendor': u'Red Hat, Inc.', 'pdevice_id': u'7010', 'pclass': u'IDE interface', 'psdevice': u'Qemu virtual machine', 'sriov_vfs_pci_address': u'', 'pvendor': u'Intel Corporation', 'pvendor_id': u'8086', 'pclass_id': u'010180', 'sriov_totalvfs': None} 2019-01-28 17:07:17.797 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] attr: {'sriov_numvfs': 0, 'driver': u'uhci_hcd', 'psvendor': u'Red Hat, Inc.', 'pdevice_id': u'7020', 'pclass': u'USB controller', 'psdevice': u'QEMU Virtual Machine', 'sriov_vfs_pci_address': u'', 'pvendor': u'Intel Corporation', 'pvendor_id': u'8086', 'pclass_id': u'0c0300', 'sriov_totalvfs': None} 2019-01-28 17:07:17.810 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] attr: {'sriov_numvfs': 0, 'driver': None, 'psvendor': u'QEMU Virtual Machine', 'pdevice_id': u'00b8', 'pclass': u'VGA compatible controller', 'psdevice': u'QEMU Virtual Machine', 'sriov_vfs_pci_address': u'', 'pvendor': u'Cirrus Logic', 'pvendor_id': u'1013', 'pclass_id': u'030000', 'sriov_totalvfs': None} 2019-01-28 17:07:17.823 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] attr: {'sriov_numvfs': 0, 'driver': u'ahci', 'psvendor': u'-p01', 'pdevice_id': u'2922', 'pclass': u'SATA controller', 'psdevice': u'Red Hat, Inc.', 'sriov_vfs_pci_address': u'', 'pvendor': u'Intel Corporation', 'pvendor_id': u'8086', 'pclass_id': u'010601', 'sriov_totalvfs': None} 2019-01-28 17:07:17.836 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] attr: {'sriov_numvfs': 0, 'driver': u'virtio-pci', 'psvendor': u'Red Hat, Inc.', 'pdevice_id': u'1002', 'pclass': u'Unclassified device [00ff]', 'psdevice': u'Device 0005', 'sriov_vfs_pci_address': u'', 'pvendor': u'Red Hat, Inc.', 'pvendor_id': u'1af4', 'pclass_id': u'00ff00', 'sriov_totalvfs': None} 2019-01-28 17:07:17.883 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] Already in db numa_node=0 mynuma_nodes=[0] 2019-01-28 17:07:17.915 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] Logical CPU topology: host:controller-0 (controller,worker), sockets:1, cores/socket=6, threads/core=1, reference:current (unchanged) 2019-01-28 17:07:17.915 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] cpu_id : 0 1 2 3 4 5 2019-01-28 17:07:17.915 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] socket_id : 0 0 0 0 0 0 2019-01-28 17:07:17.915 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] core_id : 0 1 2 3 4 5 2019-01-28 17:07:17.915 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] thread_id : 0 0 0 0 0 0 2019-01-28 17:07:17.921 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] update_cpu_config, host uuid: (3a53c151-5f31-42c5-9224-b754ff15d44c), force: (False) 2019-01-28 17:07:17.937 20749 WARNING sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] controller-0: iconfig out of date: target 7bf94fa8-a869-4138-8416-2f8fe90b5b9f, applied 580a423d-c5dc-4f7f-a45c-2cbee5587aff 2019-01-28 17:07:17.937 20749 WARNING sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] SYS_I Raise system config alarm: host controller-0 config applied: 580a423d-c5dc-4f7f-a45c-2cbee5587aff vs. target: 7bf94fa8-a869-4138-8416-2f8fe90b5b9f. 2019-01-28 17:07:17.948 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] _config_update_hosts config_uuid=7bf94fa8-a869-4138-8416-2f8fe90b5b9f 2019-01-28 17:07:22.144 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] platform_interfaces host_id=1 info_list=[] 2019-01-28 17:07:22.186 32701 INFO sysinv.agent.manager [-] iscsi initiator name = iqn.1994-05.com.redhat:80988ace3499 2019-01-28 17:07:22.196 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] controller-0 updating iscsi initiator=iqn.1994-05.com.redhat:80988ace3499 2019-01-28 17:07:22.248 20749 WARNING sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] controller-0: iconfig out of date: target 7bf94fa8-a869-4138-8416-2f8fe90b5b9f, applied install 2019-01-28 17:07:22.248 20749 WARNING sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] SYS_I Raise system config alarm: host controller-0 config applied: install vs. target: 7bf94fa8-a869-4138-8416-2f8fe90b5b9f. 2019-01-28 17:07:22.254 32701 INFO sysinv.agent.manager [-] Sysinv Agent platform update by host: {'config_applied': 'install', 'availability': 'available', 'iscsi_initiator_name': 'iqn.1994-05.com.redhat:80988ace3499'} 2019-01-28 17:07:24.839 32701 INFO sysinv.agent.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] config_apply_runtime_manifest: 7bf94fa8-a869-4138-8416-2f8fe90b5b9f {u'classes': [u'platform::compute::grub::runtime'], u'force': False, u'personalities': [u'controller', u'worker'], u'host_uuids': [u'3a53c151-5f31-42c5-9224-b754ff15d44c']} controller 2019-01-28 17:07:24.840 32701 INFO sysinv.agent.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] controller-active 2019-01-28 17:07:24.840 32701 INFO sysinv.agent.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] _apply_runtime_manifest with hieradata_path = '/opt/platform/puppet/19.01/hieradata' 2019-01-28 17:07:32.946 32701 INFO sysinv.agent.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] Agent config applied 7bf94fa8-a869-4138-8416-2f8fe90b5b9f 2019-01-28 17:07:32.979 20749 INFO sysinv.conductor.manager [req-fba0810d-c2d0-4e56-adbd-b76ee4bb9170 None None] SYS_I Clear system config alarm: controller-0 2019-01-28 17:08:11.351 21006 WARNING sysinv.api.controllers.v1.dns [-] dns {'uuid': u'd9b17baa-0ac2-41cf-8524-9184aa1cf8cf', 'nameservers': u'10.248.2.1', 'created_at': datetime.datetime(2019, 1, 28, 16, 48, 14, 690469, tzinfo=), 'updated_at': datetime.datetime(2019, 1, 28, 16, 49, 2, 67578, tzinfo=), 'action': None, 'isystem_uuid': u'f1ccce70-4b53-4a09-a9fb-803d187a8318', 'id': 1, 'forisystemid': 1} 2019-01-28 17:08:11.398 21006 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:08:11.524 20749 WARNING sysinv.conductor.manager [req-96f2a91f-be40-4a9f-855b-cbf1f89a87f1 admin admin] controller-0: iconfig out of date: target 37d06ca9-fd44-4fb6-97ec-15341ac1bc49, applied 7bf94fa8-a869-4138-8416-2f8fe90b5b9f 2019-01-28 17:08:11.524 20749 WARNING sysinv.conductor.manager [req-96f2a91f-be40-4a9f-855b-cbf1f89a87f1 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 7bf94fa8-a869-4138-8416-2f8fe90b5b9f vs. target: 37d06ca9-fd44-4fb6-97ec-15341ac1bc49. 2019-01-28 17:08:11.545 20749 INFO sysinv.conductor.manager [req-96f2a91f-be40-4a9f-855b-cbf1f89a87f1 admin admin] _config_update_hosts config_uuid=37d06ca9-fd44-4fb6-97ec-15341ac1bc49 2019-01-28 17:08:12.379 32701 INFO sysinv.agent.manager [req-93f21ffe-b3de-49d9-bf2a-39df193d60dd admin None] config_apply_runtime_manifest: 7bf94fa8-a869-4138-8416-2f8fe90b5b9f {u'classes': [u'openstack::keystone::endpoint::runtime'], u'force': False, u'personalities': [u'controller'], u'host_uuids': u'3a53c151-5f31-42c5-9224-b754ff15d44c'} controller 2019-01-28 17:08:12.380 32701 INFO sysinv.agent.manager [req-93f21ffe-b3de-49d9-bf2a-39df193d60dd admin None] controller-active 2019-01-28 17:08:12.380 32701 INFO sysinv.agent.manager [req-93f21ffe-b3de-49d9-bf2a-39df193d60dd admin None] _apply_runtime_manifest with hieradata_path = '/opt/platform/puppet/19.01/hieradata' 2019-01-28 17:08:12.385 20749 INFO sysinv.conductor.manager [-] _controller_config_active_apply about to resize the filesystem 2019-01-28 17:08:12.385 20749 WARNING sysinv.conductor.manager [-] resizing filesystems 2019-01-28 17:08:12.414 20749 INFO sysinv.conductor.manager [-] Performed drbdadm resize all 2019-01-28 17:08:12.500 20749 INFO sysinv.conductor.manager [-] drbd-overview: pgsql-40.0, cgcs-9.9, extension-0.96875, patch-vault-0, etcd-4.8, dockerdistribution-16.0 2019-01-28 17:08:12.501 20749 INFO sysinv.conductor.manager [-] lvdisplay: pgsql-40.0, cgcs-10.0, extension-1.0, patch-vault-0, etcd-5.0, dockerdistribution-16.0 2019-01-28 17:08:12.723 20749 INFO sysinv.conductor.manager [-] SYS_I Clear system config alarm: controller-0 2019-01-28 17:08:35.484 21006 WARNING sysinv.api.controllers.v1.ntp [-] ntp {'uuid': u'd9b17baa-0ac2-41cf-8524-9184aa1cf8cf', 'created_at': datetime.datetime(2019, 1, 28, 16, 48, 14, 697432, tzinfo=), 'enabled': True, 'updated_at': None, 'ntpservers': u'0.pool.ntp.org,1.pool.ntp.org', 'action': None, 'isystem_uuid': u'f1ccce70-4b53-4a09-a9fb-803d187a8318', 'id': 1, 'forisystemid': 1} 2019-01-28 17:08:35.536 20749 WARNING sysinv.conductor.manager [req-6da54d72-cf40-4590-a588-1b7e9615d938 admin admin] controller-0: iconfig out of date: target d10ed7b0-b60e-4217-9be8-85b3b1b16ac4, applied 7bf94fa8-a869-4138-8416-2f8fe90b5b9f 2019-01-28 17:08:35.536 20749 WARNING sysinv.conductor.manager [req-6da54d72-cf40-4590-a588-1b7e9615d938 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 7bf94fa8-a869-4138-8416-2f8fe90b5b9f vs. target: d10ed7b0-b60e-4217-9be8-85b3b1b16ac4. 2019-01-28 17:08:35.549 20749 INFO sysinv.conductor.manager [req-6da54d72-cf40-4590-a588-1b7e9615d938 admin admin] _config_update_hosts config_uuid=d10ed7b0-b60e-4217-9be8-85b3b1b16ac4 2019-01-28 17:08:53.642 20749 WARNING sysinv.conductor.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] controller-0: iconfig out of date: target a53c42e2-0486-40a5-a91a-38795ec80200, applied 7bf94fa8-a869-4138-8416-2f8fe90b5b9f 2019-01-28 17:08:53.643 20749 WARNING sysinv.conductor.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 7bf94fa8-a869-4138-8416-2f8fe90b5b9f vs. target: a53c42e2-0486-40a5-a91a-38795ec80200. 2019-01-28 17:08:53.645 20749 INFO sysinv.conductor.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] _config_update_hosts config_uuid=253c42e2-0486-40a5-a91a-38795ec80200 2019-01-28 17:09:10.649 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_partition) 2019-01-28 17:09:10.655 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_partition) 2019-01-28 17:10:10.723 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_partition) 2019-01-28 17:10:10.729 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_partition) 2019-01-28 17:10:20.954 32701 INFO sysinv.agent.manager [req-96f2a91f-be40-4a9f-855b-cbf1f89a87f1 admin admin] Agent config applied 37d06ca9-fd44-4fb6-97ec-15341ac1bc49 2019-01-28 17:10:20.979 20749 WARNING sysinv.conductor.manager [req-96f2a91f-be40-4a9f-855b-cbf1f89a87f1 admin admin] controller-0: iconfig out of date: target a53c42e2-0486-40a5-a91a-38795ec80200, applied 37d06ca9-fd44-4fb6-97ec-15341ac1bc49 2019-01-28 17:10:20.979 20749 WARNING sysinv.conductor.manager [req-96f2a91f-be40-4a9f-855b-cbf1f89a87f1 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 37d06ca9-fd44-4fb6-97ec-15341ac1bc49 vs. target: a53c42e2-0486-40a5-a91a-38795ec80200. 2019-01-28 17:10:20.992 32701 INFO sysinv.agent.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] config_apply_runtime_manifest: 253c42e2-0486-40a5-a91a-38795ec80200 {u'force': False, u'host_uuids': u'3a53c151-5f31-42c5-9224-b754ff15d44c', u'partition_uuid': u'c0c46c7d-bae2-44c7-95d5-fd832c229500', u'idisk_uuid': u'18101309-ae81-4916-867f-d33f45943f28', u'classes': [u'platform::partitions::runtime'], u'report_status': u'manage_disk_partitions', u'personalities': [u'controller']} controller 2019-01-28 17:10:20.993 32701 INFO sysinv.agent.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] controller-active 2019-01-28 17:10:20.993 32701 INFO sysinv.agent.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] _apply_runtime_manifest with hieradata_path = '/opt/platform/puppet/19.01/hieradata' 2019-01-28 17:10:29.304 29542 INFO manage-partitions [-] Called partition 'check' with 'None' 'None' and '[{"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part1", "start_mib": 1, "uuid": "491211cd-f682-41ca-bc32-67f019b7b95b", "device_node": "/dev/sda1", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "21686148-6449-6e6f-744e-656564454649", "size_mib": 1}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part2", "start_mib": 2, "uuid": "24932520-86e1-4917-a18f-f5b26d88040c", "device_node": "/dev/sda2", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 500}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part3", "start_mib": 502, "uuid": "e88cae09-e541-4edb-8bed-235d73800070", "device_node": "/dev/sda3", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 20000}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part4", "start_mib": 20502, "uuid": "40be4c0d-7110-434b-819d-cbb92db19b04", "device_node": "/dev/sda4", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "e6d6d379-f507-44c2-a23c-238f2a3df928", "size_mib": 224256}]' 2019-01-28 17:10:29.304 29542 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:29.310 29542 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:29.542 29542 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:29.546 29542 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:29.773 29542 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:29.776 29542 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:29.992 29542 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:29.996 29542 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:30.220 29542 INFO manage-partitions [-] Executing command: 'sgdisk -i 1 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:30.240 29542 INFO manage-partitions [-] Executing command: 'sgdisk -i 2 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:30.250 29542 INFO manage-partitions [-] Executing command: 'sgdisk -i 3 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:30.259 29542 INFO manage-partitions [-] Executing command: 'sgdisk -i 4 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:30.837 29686 INFO manage-partitions [-] Called partition 'create' with 'None' 'None' and '[{"ihost_uuid": "3a53c151-5f31-42c5-9224-b754ff15d44c", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "req_size_mib": 6144, "part_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5", "req_guid": "ba5eba11-0000-1111-2222-000000000001", "req_uuid": "c0c46c7d-bae2-44c7-95d5-fd832c229500"}]' 2019-01-28 17:10:30.838 29686 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:30.841 29686 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:31.058 29686 INFO manage-partitions [-] Executing command: 'sgdisk -i 5 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:31.083 29686 INFO manage-partitions [-] Executing pipe command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 unit mib print free' 2019-01-28 17:10:31.336 29686 INFO manage-partitions [-] Executing pipe command: 'sgdisk -p /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:31.442 29686 INFO manage-partitions [-] Executing command: 'blockdev --getss /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:31.445 29686 INFO manage-partitions [-] Executing command: 'dd bs=512 if=/dev/zero of=/dev/disk/by-path/pci-0000:00:07.0-ata-1.0 oflag=direct count=34 seek=501264384' 2019-01-28 17:10:31.469 29686 INFO manage-partitions [-] Zero-out beginning of partition. Output: 34+0 records in 34+0 records out 17408 bytes (17 kB) copied, 0.0215843 s, 807 kB/s 2019-01-28 17:10:31.469 29686 INFO manage-partitions [-] Executing command: 'dd bs=512 if=/dev/zero of=/dev/disk/by-path/pci-0000:00:07.0-ata-1.0 oflag=direct count=34 seek=513847262' 2019-01-28 17:10:31.497 29686 INFO manage-partitions [-] Zero-out end of partition. Output: 34+0 records in 34+0 records out 17408 bytes (17 kB) copied, 0.0251758 s, 691 kB/s 2019-01-28 17:10:31.497 29686 INFO manage-partitions [-] Partition details: {'sector_size': 512, 'start_in_sectors': 501264384, 'size_in_sectors': 12582912, 'part_end': 513847262, 'disk_node': u'/dev/disk/by-path/pci-0000:00:07.0-ata-1.0'} 2019-01-28 17:10:31.498 29686 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:31.503 29686 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 unit mib mkpart primary 244758 250902' 2019-01-28 17:10:31.647 29686 INFO manage-partitions [-] Executing command: 'sgdisk --typecode=5:ba5eba11-0000-1111-2222-000000000001 --change-name=5:LVM Physical Volume /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:32.776 29686 INFO manage-partitions [-] Executing command: 'ls /dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5' 2019-01-28 17:10:33.082 29686 INFO manage-partitions [-] Executing command: 'ls /dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5' 2019-01-28 17:10:33.393 29686 INFO manage-partitions [-] Executing command: 'ls /dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5' 2019-01-28 17:10:33.396 29686 INFO manage-partitions [-] Executing command: 'sgdisk -i 5 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:33.502 29686 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:33.506 29686 INFO manage-partitions [-] Executing pipe command: 'sgdisk -p /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:33.611 29686 INFO manage-partitions [-] Executing command: 'blockdev --getss /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:33.691 29686 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:10:33.695 20749 INFO sysinv.conductor.manager [req-c05ab95d-1f8c-4bc2-8e00-cf7f5eb82093 None None] PART updating information for partition c0c46c7d-bae2-44c7-95d5-fd832c229500 on host 3a53c151-5f31-42c5-9224-b754ff15d44c was successful: {u'status': 0, u'device_path': u'/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5', u'start_mib': 244758.0, u'uuid': u'c0c46c7d-bae2-44c7-95d5-fd832c229500', u'ihost_uuid': u'3a53c151-5f31-42c5-9224-b754ff15d44c', u'type_name': u'LVM Physical Volume', u'available_mib': 363496, u'end_mib': 250902.0, u'type_guid': u'ba5eba11-0000-1111-2222-000000000001', u'size_mib': 6144.0} 2019-01-28 17:10:34.248 20749 INFO sysinv.conductor.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] PART manifest application for partition c0c46c7d-bae2-44c7-95d5-fd832c229500 on host 3a53c151-5f31-42c5-9224-b754ff15d44c was successful 2019-01-28 17:10:34.253 32701 INFO sysinv.agent.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] Agent config applied 253c42e2-0486-40a5-a91a-38795ec80200 2019-01-28 17:10:34.274 20749 WARNING sysinv.conductor.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] controller-0: iconfig out of date: target a53c42e2-0486-40a5-a91a-38795ec80200, applied 253c42e2-0486-40a5-a91a-38795ec80200 2019-01-28 17:10:34.275 20749 WARNING sysinv.conductor.manager [req-fe27a0c2-b3ad-4c10-beaf-7488194d3a24 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 253c42e2-0486-40a5-a91a-38795ec80200 vs. target: a53c42e2-0486-40a5-a91a-38795ec80200. 2019-01-28 17:10:45.452 20749 WARNING sysinv.conductor.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] controller-0: iconfig out of date: target 8a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50, applied 253c42e2-0486-40a5-a91a-38795ec80200 2019-01-28 17:10:45.453 20749 WARNING sysinv.conductor.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 253c42e2-0486-40a5-a91a-38795ec80200 vs. target: 8a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50. 2019-01-28 17:10:45.455 20749 INFO sysinv.conductor.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] _config_update_hosts config_uuid=0a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50 2019-01-28 17:10:46.913 32701 INFO sysinv.agent.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] config_apply_runtime_manifest: 0a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50 {u'force': False, u'host_uuids': u'3a53c151-5f31-42c5-9224-b754ff15d44c', u'partition_uuid': u'd9826c60-d26c-4e3e-9b58-7dc31af211dd', u'idisk_uuid': u'18101309-ae81-4916-867f-d33f45943f28', u'classes': [u'platform::partitions::runtime'], u'report_status': u'manage_disk_partitions', u'personalities': [u'controller']} controller 2019-01-28 17:10:46.913 32701 INFO sysinv.agent.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] controller-active 2019-01-28 17:10:46.913 32701 INFO sysinv.agent.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] _apply_runtime_manifest with hieradata_path = '/opt/platform/puppet/19.01/hieradata' 2019-01-28 17:10:55.097 3500 INFO manage-partitions [-] Called partition 'check' with 'None' 'None' and '[{"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part1", "start_mib": 1, "uuid": "491211cd-f682-41ca-bc32-67f019b7b95b", "device_node": "/dev/sda1", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "21686148-6449-6e6f-744e-656564454649", "size_mib": 1}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part2", "start_mib": 2, "uuid": "24932520-86e1-4917-a18f-f5b26d88040c", "device_node": "/dev/sda2", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 500}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part3", "start_mib": 502, "uuid": "e88cae09-e541-4edb-8bed-235d73800070", "device_node": "/dev/sda3", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 20000}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part4", "start_mib": 20502, "uuid": "40be4c0d-7110-434b-819d-cbb92db19b04", "device_node": "/dev/sda4", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "e6d6d379-f507-44c2-a23c-238f2a3df928", "size_mib": 224256}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5", "start_mib": 244758, "uuid": "c0c46c7d-bae2-44c7-95d5-fd832c229500", "device_node": "/dev/sda5", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ba5eba11-0000-1111-2222-000000000001", "size_mib": 6144}]' 2019-01-28 17:10:55.098 3500 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:55.101 3500 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:55.324 3500 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:55.328 3500 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:55.547 3500 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:55.558 3500 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:55.810 3500 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:55.823 3500 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:56.075 3500 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:56.089 3500 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:56.332 3500 INFO manage-partitions [-] Executing command: 'sgdisk -i 1 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:56.339 3500 INFO manage-partitions [-] Executing command: 'sgdisk -i 2 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:56.348 3500 INFO manage-partitions [-] Executing command: 'sgdisk -i 3 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:56.358 3500 INFO manage-partitions [-] Executing command: 'sgdisk -i 4 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:56.367 3500 INFO manage-partitions [-] Executing command: 'sgdisk -i 5 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:56.968 3691 INFO manage-partitions [-] Called partition 'create' with 'None' 'None' and '[{"ihost_uuid": "3a53c151-5f31-42c5-9224-b754ff15d44c", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "req_size_mib": 34816, "part_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part6", "req_guid": "ba5eba11-0000-1111-2222-000000000001", "req_uuid": "d9826c60-d26c-4e3e-9b58-7dc31af211dd"}]' 2019-01-28 17:10:56.968 3691 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:56.972 3691 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:10:57.196 3691 INFO manage-partitions [-] Executing command: 'sgdisk -i 6 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:57.201 3691 INFO manage-partitions [-] Executing pipe command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 unit mib print free' 2019-01-28 17:10:57.431 3691 INFO manage-partitions [-] Executing pipe command: 'sgdisk -p /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:57.537 3691 INFO manage-partitions [-] Executing command: 'blockdev --getss /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:57.540 3691 INFO manage-partitions [-] Executing command: 'dd bs=512 if=/dev/zero of=/dev/disk/by-path/pci-0000:00:07.0-ata-1.0 oflag=direct count=34 seek=513847296' 2019-01-28 17:10:57.546 3691 INFO manage-partitions [-] Zero-out beginning of partition. Output: 34+0 records in 34+0 records out 17408 bytes (17 kB) copied, 0.00314294 s, 5.5 MB/s 2019-01-28 17:10:57.546 3691 INFO manage-partitions [-] Executing command: 'dd bs=512 if=/dev/zero of=/dev/disk/by-path/pci-0000:00:07.0-ata-1.0 oflag=direct count=34 seek=585150430' 2019-01-28 17:10:57.570 3691 INFO manage-partitions [-] Zero-out end of partition. Output: 34+0 records in 34+0 records out 17408 bytes (17 kB) copied, 0.0200552 s, 868 kB/s 2019-01-28 17:10:57.570 3691 INFO manage-partitions [-] Partition details: {'sector_size': 512, 'start_in_sectors': 513847296, 'size_in_sectors': 71303168, 'part_end': 585150430, 'disk_node': u'/dev/disk/by-path/pci-0000:00:07.0-ata-1.0'} 2019-01-28 17:10:57.571 3691 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:57.577 3691 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 unit mib mkpart primary 250902 285718' 2019-01-28 17:10:57.714 3691 INFO manage-partitions [-] Executing command: 'sgdisk --typecode=6:ba5eba11-0000-1111-2222-000000000001 --change-name=6:LVM Physical Volume /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:58.845 3691 INFO manage-partitions [-] Executing command: 'ls /dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part6' 2019-01-28 17:10:59.148 3691 INFO manage-partitions [-] Executing command: 'ls /dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part6' 2019-01-28 17:10:59.451 3691 INFO manage-partitions [-] Executing command: 'ls /dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part6' 2019-01-28 17:10:59.454 3691 INFO manage-partitions [-] Executing command: 'sgdisk -i 6 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:59.559 3691 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:59.563 3691 INFO manage-partitions [-] Executing pipe command: 'sgdisk -p /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:59.669 3691 INFO manage-partitions [-] Executing command: 'blockdev --getss /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:10:59.753 3691 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:10:59.757 20749 INFO sysinv.conductor.manager [req-24cc9136-6c89-4282-8f6c-517b4a23688d None None] PART updating information for partition d9826c60-d26c-4e3e-9b58-7dc31af211dd on host 3a53c151-5f31-42c5-9224-b754ff15d44c was successful: {u'status': 0, u'device_path': u'/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part6', u'start_mib': 250902.0, u'uuid': u'd9826c60-d26c-4e3e-9b58-7dc31af211dd', u'ihost_uuid': u'3a53c151-5f31-42c5-9224-b754ff15d44c', u'type_name': u'LVM Physical Volume', u'available_mib': 328680, u'end_mib': 285718.0, u'type_guid': u'ba5eba11-0000-1111-2222-000000000001', u'size_mib': 34816.0} 2019-01-28 17:11:00.334 20749 INFO sysinv.conductor.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] PART manifest application for partition d9826c60-d26c-4e3e-9b58-7dc31af211dd on host 3a53c151-5f31-42c5-9224-b754ff15d44c was successful 2019-01-28 17:11:00.340 32701 INFO sysinv.agent.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] Agent config applied 0a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50 2019-01-28 17:11:00.359 20749 WARNING sysinv.conductor.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] controller-0: iconfig out of date: target 8a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50, applied 0a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50 2019-01-28 17:11:00.359 20749 WARNING sysinv.conductor.manager [req-4fbfd933-ddba-4f9e-8ab9-e96bc3139c94 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 0a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50 vs. target: 8a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50. 2019-01-28 17:11:40.676 21006 INFO sysinv.api.controllers.v1.lvg [-] Updated lvg capabilities={'concurrent_disk_operations': 2, 'instance_backing': 'image'} 2019-01-28 17:12:10.796 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg) 2019-01-28 17:12:10.802 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg) 2019-01-28 17:12:19.900 32701 INFO sysinv.agent.manager [-] interface ens4 enabled to receive LLDP PDUs 2019-01-28 17:12:19.904 32701 INFO sysinv.agent.manager [-] interface eth1000 enabled to receive LLDP PDUs 2019-01-28 17:12:19.909 32701 INFO sysinv.agent.manager [-] interface eth1001 enabled to receive LLDP PDUs 2019-01-28 17:12:50.109 32701 ERROR sysinv.agent.lldp.drivers.ovs.driver [-] Failed to run command ovs-vsctl --timeout 10 --format json --columns name,ports list Bridge: error: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) 2019-01-28 17:12:50.155 32701 ERROR sysinv.agent.lldp.drivers.ovs.driver [-] Failed to run command ovs-vsctl --timeout 10 --format json --columns name,ports list Bridge: error: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) 2019-01-28 17:12:50.226 20749 INFO sysinv.conductor.manager [req-28fac56c-8615-4b35-8d63-0b60caae0409 None None] Attempting to create new LLDP agent {'status': u'rx=enabled,tx=enabled', 'host_id': 1, 'port_id': 2} on host 1 2019-01-28 17:12:50.471 32701 INFO sysinv.agent.manager [-] interface ens4 disabled after querying LLDP neighbors 2019-01-28 17:12:50.477 32701 INFO sysinv.agent.manager [-] interface eth1000 disabled after querying LLDP neighbors 2019-01-28 17:12:50.483 32701 INFO sysinv.agent.manager [-] interface eth1001 disabled after querying LLDP neighbors 2019-01-28 17:13:10.829 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg) 2019-01-28 17:13:10.835 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg) 2019-01-28 17:13:23.898 21006 WARNING wsme.api [-] Client-side error: No disk with id c0c46c7d-bae2-44c7-95d5-fd832c229500 2019-01-28 17:13:24.034 20749 WARNING sysinv.conductor.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] controller-0: iconfig out of date: target f09f4cec-8968-47d2-a7a1-5f238daa05b2, applied 0a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50 2019-01-28 17:13:24.034 20749 WARNING sysinv.conductor.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 0a03f8bd-0eb7-47f0-8f9a-db0cebcfdc50 vs. target: f09f4cec-8968-47d2-a7a1-5f238daa05b2. 2019-01-28 17:13:24.036 20749 INFO sysinv.conductor.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] _config_update_hosts config_uuid=709f4cec-8968-47d2-a7a1-5f238daa05b2 2019-01-28 17:13:25.568 32701 INFO sysinv.agent.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] config_apply_runtime_manifest: 709f4cec-8968-47d2-a7a1-5f238daa05b2 {u'classes': [u'platform::lvm::controller::runtime'], u'force': False, u'personalities': [u'controller']} controller 2019-01-28 17:13:25.569 32701 INFO sysinv.agent.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] controller-active 2019-01-28 17:13:25.569 32701 INFO sysinv.agent.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] _apply_runtime_manifest with hieradata_path = '/opt/platform/puppet/19.01/hieradata' 2019-01-28 17:13:33.971 32701 INFO sysinv.agent.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] Agent config applied 709f4cec-8968-47d2-a7a1-5f238daa05b2 2019-01-28 17:13:34.031 20749 WARNING sysinv.conductor.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] controller-0: iconfig out of date: target f09f4cec-8968-47d2-a7a1-5f238daa05b2, applied 709f4cec-8968-47d2-a7a1-5f238daa05b2 2019-01-28 17:13:34.032 20749 WARNING sysinv.conductor.manager [req-9dd3e8dc-9fd2-4e30-bfcc-b65c4e76e213 admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 709f4cec-8968-47d2-a7a1-5f238daa05b2 vs. target: f09f4cec-8968-47d2-a7a1-5f238daa05b2. 2019-01-28 17:13:34.254 21006 WARNING wsme.api [-] Client-side error: No disk with id d9826c60-d26c-4e3e-9b58-7dc31af211dd 2019-01-28 17:14:10.819 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg, audit_pv) 2019-01-28 17:14:10.825 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg, audit_pv) 2019-01-28 17:14:12.405 20749 INFO sysinv.conductor.manager [req-0867de79-5ee6-429a-9b51-a6c1cc9e91bd None None] PV uuid: d94a0552-907b-42eb-83e9-1ced97f2c8cd changed UUID from None to YZMvyt-rw6G-6nkh-59JI-liOC-54Yb-926FfH 2019-01-28 17:14:12.429 20749 INFO sysinv.conductor.manager [req-0867de79-5ee6-429a-9b51-a6c1cc9e91bd None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:15:10.874 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg, audit_pv) 2019-01-28 17:15:10.880 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg, audit_pv) 2019-01-28 17:15:12.562 20749 INFO sysinv.conductor.manager [req-ca04c658-fa0e-48e6-8437-e6073376d3ae None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:15:19.905 32701 INFO sysinv.agent.manager [-] interface ens4 enabled to receive LLDP PDUs 2019-01-28 17:15:19.911 32701 INFO sysinv.agent.manager [-] interface eth1000 enabled to receive LLDP PDUs 2019-01-28 17:15:19.915 32701 INFO sysinv.agent.manager [-] interface eth1001 enabled to receive LLDP PDUs 2019-01-28 17:15:35.164 21006 INFO sysinv.api.controllers.v1.controller_fs [-] _get_controller_cgtsvg_limit cgtsvg0_free_mib=28800, cgtsvg1_free_mib=0 2019-01-28 17:15:35.164 21006 INFO sysinv.api.controllers.v1.controller_fs [-] SYS_I filesystem limits cgtsvg0_free_mib=28800, cgtsvg1_free_mib=0, cgtsvg_max_free_GiB=28 2019-01-28 17:15:35.164 21006 INFO sysinv.api.controllers.v1.controller_fs [-] _check_controller_fs ceph_mon_gib_new = 20 2019-01-28 17:15:35.164 21006 INFO sysinv.api.controllers.v1.controller_fs [-] _check_controller_fs cgtsvg_growth_gib = 20 2019-01-28 17:15:35.164 21006 INFO sysinv.api.controllers.v1.controller_fs [-] _check_controller_fs cgtsvg_max_free_GiB = 28 2019-01-28 17:15:36.009 21006 INFO sysinv.api.controllers.v1.controller_fs [-] _total_size_controller_fs total filesysem size 185 2019-01-28 17:15:36.010 21006 INFO sysinv.api.controllers.v1.controller_fs [-] _check_controller_fs rootfs_configured_size_GiB = 205 2019-01-28 17:15:36.227 21006 INFO sysinv.api.controllers.v1.ceph_mon [-] Creating ceph-mon DB entry for host uuid 3a53c151-5f31-42c5-9224-b754ff15d44c: {'forihostid': 1, 'device_path': None, 'uuid': '94f371ce-6ba8-4dfa-82bc-0b73fd9c2813', 'ceph_mon_gib': 20, 'created_at': None, 'hostname': None, 'updated_at': None, 'ihost_uuid': u'3a53c151-5f31-42c5-9224-b754ff15d44c', 'state': u'configured', 'task': None, 'ceph_mon_dev': None, 'id': None} 2019-01-28 17:15:36.480 20749 WARNING sysinv.conductor.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] controller-0: iconfig out of date: target daeec83f-c5a4-4c66-b919-b01ee23c6bcb, applied 709f4cec-8968-47d2-a7a1-5f238daa05b2 2019-01-28 17:15:36.480 20749 WARNING sysinv.conductor.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 709f4cec-8968-47d2-a7a1-5f238daa05b2 vs. target: daeec83f-c5a4-4c66-b919-b01ee23c6bcb. 2019-01-28 17:15:36.482 20749 INFO sysinv.conductor.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] _config_update_hosts config_uuid=5aeec83f-c5a4-4c66-b919-b01ee23c6bcb 2019-01-28 17:15:50.069 32701 ERROR sysinv.agent.lldp.drivers.ovs.driver [-] Failed to run command ovs-vsctl --timeout 10 --format json --columns name,ports list Bridge: error: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) 2019-01-28 17:15:50.120 32701 ERROR sysinv.agent.lldp.drivers.ovs.driver [-] Failed to run command ovs-vsctl --timeout 10 --format json --columns name,ports list Bridge: error: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) 2019-01-28 17:15:50.292 32701 INFO sysinv.agent.manager [-] interface ens4 disabled after querying LLDP neighbors 2019-01-28 17:15:50.301 32701 INFO sysinv.agent.manager [-] interface eth1000 disabled after querying LLDP neighbors 2019-01-28 17:15:50.309 32701 INFO sysinv.agent.manager [-] interface eth1001 disabled after querying LLDP neighbors 2019-01-28 17:15:51.617 32701 INFO sysinv.agent.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] config_apply_runtime_manifest: 5aeec83f-c5a4-4c66-b919-b01ee23c6bcb {u'classes': [u'platform::partitions::runtime', u'platform::lvm::controller::runtime', u'platform::haproxy::runtime', u'openstack::keystone::endpoint::runtime', u'platform::filesystem::img_conversions::runtime', u'platform::ceph::runtime', u'platform::sm::norestart::runtime'], u'force': False, u'personalities': [u'controller'], u'report_status': u'ceph_config', u'host_uuids': []} controller 2019-01-28 17:15:51.617 32701 INFO sysinv.agent.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] controller-active 2019-01-28 17:15:51.617 32701 INFO sysinv.agent.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] _apply_runtime_manifest with hieradata_path = '/opt/platform/puppet/19.01/hieradata' 2019-01-28 17:16:10.899 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg, audit_pv) 2019-01-28 17:16:10.908 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg, audit_pv) 2019-01-28 17:16:11.958 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-16-11 patch 2019-01-28 17:16:11.958 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_end. No changes from mtce/1.0. 2019-01-28 17:17:10.948 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg, audit_pv) 2019-01-28 17:17:10.955 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg, audit_pv) 2019-01-28 17:18:11.010 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg, audit_pv) 2019-01-28 17:18:11.016 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg, audit_pv) 2019-01-28 17:18:51.446 3306 INFO manage-partitions [-] Called partition 'check' with 'None' 'None' and '[{"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part1", "start_mib": 1, "uuid": "491211cd-f682-41ca-bc32-67f019b7b95b", "device_node": "/dev/sda1", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "21686148-6449-6e6f-744e-656564454649", "size_mib": 1}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part2", "start_mib": 2, "uuid": "24932520-86e1-4917-a18f-f5b26d88040c", "device_node": "/dev/sda2", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 500}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part3", "start_mib": 502, "uuid": "e88cae09-e541-4edb-8bed-235d73800070", "device_node": "/dev/sda3", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 20000}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part4", "start_mib": 20502, "uuid": "40be4c0d-7110-434b-819d-cbb92db19b04", "device_node": "/dev/sda4", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "e6d6d379-f507-44c2-a23c-238f2a3df928", "size_mib": 224256}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5", "start_mib": 244758, "uuid": "c0c46c7d-bae2-44c7-95d5-fd832c229500", "device_node": "/dev/sda5", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ba5eba11-0000-1111-2222-000000000001", "size_mib": 6144}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part6", "start_mib": 250902, "uuid": "d9826c60-d26c-4e3e-9b58-7dc31af211dd", "device_node": "/dev/sda6", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ba5eba11-0000-1111-2222-000000000001", "size_mib": 34816}]' 2019-01-28 17:18:51.447 3306 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:51.450 3306 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:18:51.672 3306 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:51.677 3306 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:18:51.913 3306 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:51.923 3306 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:18:52.176 3306 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:52.187 3306 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:18:52.441 3306 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:52.453 3306 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:18:52.703 3306 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:52.718 3306 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:18:52.957 3306 INFO manage-partitions [-] Executing command: 'sgdisk -i 1 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:52.977 3306 INFO manage-partitions [-] Executing command: 'sgdisk -i 2 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:52.992 3306 INFO manage-partitions [-] Executing command: 'sgdisk -i 3 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:53.002 3306 INFO manage-partitions [-] Executing command: 'sgdisk -i 4 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:53.008 3306 INFO manage-partitions [-] Executing command: 'sgdisk -i 5 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:18:53.014 3306 INFO manage-partitions [-] Executing command: 'sgdisk -i 6 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:19:10.996 32701 WARNING sysinv.openstack.common.loopingcall [-] task run outlasted interval by 171.289478 sec 2019-01-28 17:19:11.008 20749 INFO sysinv.conductor.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] Ceph manifests success on host: 3a53c151-5f31-42c5-9224-b754ff15d44c 2019-01-28 17:19:11.054 20749 INFO sysinv.common.ceph [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] Updating crushmap with: /etc/sysinv/crushmap-aio-sx.txt 2019-01-28 17:19:11.535 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg, audit_pv) 2019-01-28 17:19:11.541 32701 INFO sysinv.agent.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] Agent config applied 5aeec83f-c5a4-4c66-b919-b01ee23c6bcb 2019-01-28 17:19:11.543 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg, audit_pv) 2019-01-28 17:19:11.565 20749 WARNING sysinv.conductor.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] controller-0: iconfig out of date: target daeec83f-c5a4-4c66-b919-b01ee23c6bcb, applied 5aeec83f-c5a4-4c66-b919-b01ee23c6bcb 2019-01-28 17:19:11.566 20749 WARNING sysinv.conductor.manager [req-6da85cbe-37c2-4c8a-8722-889b8422271a admin admin] SYS_I Raise system config alarm: host controller-0 config applied: 5aeec83f-c5a4-4c66-b919-b01ee23c6bcb vs. target: daeec83f-c5a4-4c66-b919-b01ee23c6bcb. 2019-01-28 17:19:13.298 20749 INFO sysinv.conductor.manager [req-825481e8-7dc2-446b-880a-0c542dac4fcb None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:19:14.972 20749 INFO sysinv.conductor.manager [req-cceab1b9-1033-4686-aac9-bc80961c7572 None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:19:16.503 20749 INFO sysinv.conductor.manager [req-63d614ff-5b00-4ac7-b221-31978fec9ba9 None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:19:18.233 20749 INFO sysinv.conductor.manager [req-d0e6db4b-7280-40a3-ba45-5c607bc0375e None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:19:19.918 20749 INFO sysinv.conductor.manager [req-6de8dd33-df49-47e0-b050-6ed30bfa3978 None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:20:11.145 20749 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg, audit_pv) 2019-01-28 17:20:11.153 20749 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg, audit_pv) 2019-01-28 17:20:11.305 32701 INFO sysinv.agent.manager [-] interface ens4 enabled to receive LLDP PDUs 2019-01-28 17:20:11.313 32701 INFO sysinv.agent.manager [-] interface eth1000 enabled to receive LLDP PDUs 2019-01-28 17:20:11.318 32701 INFO sysinv.agent.manager [-] interface eth1001 enabled to receive LLDP PDUs 2019-01-28 17:20:39.882 21006 INFO sysinv.api.controllers.v1.interface [-] _update_interface_datanetworks interface={'ifcapabilities': {}, 'ipv4_mode': 'disabled', 'sriov_numvfs': 0, 'used_by': [], 'schedpolicy': None, 'updated_at': None, 'txhashpolicy': None, 'ihost_uuid': u'3a53c151-5f31-42c5-9224-b754ff15d44c', 'imtu': 1500, 'uses': [], 'imac': u'52:54:00:3b:b7:9c', 'ipv6_pool': None, 'ipv6_mode': 'disabled', 'id': 3, 'vlan_id': None, 'forihostid': 1, 'iftype': u'ethernet', 'uuid': u'51fb4d6d-8304-4975-8bb4-d482379605c2', 'ipv4_pool': None, 'created_at': datetime.datetime(2019, 1, 28, 16, 49, 5, 827524, tzinfo=), 'vlan_type': None, 'ifclass': u'data', 'networks': [], 'datanetworks': [], 'networktype': u'data', 'ifname': u'data0', 'ports': None, 'aemode': 'active_standby'} datanetworks=[u'physnet0'] pns=[u'physnet0'] 2019-01-28 17:20:41.462 32701 ERROR sysinv.agent.lldp.drivers.ovs.driver [-] Failed to run command ovs-vsctl --timeout 10 --format json --columns name,ports list Bridge: error: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) 2019-01-28 17:20:41.508 32701 ERROR sysinv.agent.lldp.drivers.ovs.driver [-] Failed to run command ovs-vsctl --timeout 10 --format json --columns name,ports list Bridge: error: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) 2019-01-28 17:20:41.630 21006 INFO sysinv.api.controllers.v1.interface [-] _update_interface_datanetworks interface={'ifcapabilities': {}, 'ipv4_mode': 'disabled', 'sriov_numvfs': 0, 'used_by': [], 'schedpolicy': None, 'updated_at': None, 'txhashpolicy': None, 'ihost_uuid': u'3a53c151-5f31-42c5-9224-b754ff15d44c', 'imtu': 1500, 'uses': [], 'imac': u'52:54:00:e2:a7:a4', 'ipv6_pool': None, 'ipv6_mode': 'disabled', 'id': 4, 'vlan_id': None, 'forihostid': 1, 'iftype': u'ethernet', 'uuid': u'3c47a5e9-15eb-4594-9569-4908b5571018', 'ipv4_pool': None, 'created_at': datetime.datetime(2019, 1, 28, 16, 49, 5, 911284, tzinfo=), 'vlan_type': None, 'ifclass': u'data', 'networks': [], 'datanetworks': [], 'networktype': u'data', 'ifname': u'data1', 'ports': None, 'aemode': 'active_standby'} datanetworks=[u'physnet1'] pns=[u'physnet1'] 2019-01-28 17:20:41.681 32701 INFO sysinv.agent.manager [-] interface ens4 disabled after querying LLDP neighbors 2019-01-28 17:20:41.688 32701 INFO sysinv.agent.manager [-] interface eth1000 disabled after querying LLDP neighbors 2019-01-28 17:20:41.695 32701 INFO sysinv.agent.manager [-] interface eth1001 disabled after querying LLDP neighbors 2019-01-28 17:20:44.682 20749 INFO sysinv.conductor.manager [req-32190a5d-fe85-44ca-8e0f-90e1dd1198ba None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:20:51.534 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-20-51 patch 2019-01-28 17:20:51.535 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['action'] 2019-01-28 17:20:51.547 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost check_unlock_controller 2019-01-28 17:20:51.629 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost check_unlock_worker 2019-01-28 17:20:51.685 21006 INFO sysinv.api.controllers.v1.host [-] Set VM 4K pages for host (controller-0) node (1) pages (1825280) 2019-01-28 17:20:52.229 21006 INFO sysinv.api.controllers.v1.rest_api [-] GET cmd:http://192.168.204.2:5491/v1/query_hosts/ hdr:None payload:None 2019-01-28 17:20:52.591 21006 INFO sysinv.api.controllers.v1.rest_api [-] Response={u'data': [{u'sw_version': u'19.01', u'hostname': u'controller-0', u'nodetype': u'controller', u'patch_failed': False, u'allow_insvc_patching': True, u'subfunctions': [u'controller', u'worker'], u'requires_reboot': False, u'installed': {}, u'state': u'idle', u'interim_state': False, u'secs_since_ack': 7, u'missing_pkgs': [], u'ip': u'192.168.204.3', u'stale_details': False, u'to_remove': [], u'patch_current': True}]} 2019-01-28 17:20:52.591 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 _semantic_mtc_check_action unlock 2019-01-28 17:20:52.591 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 action=unlock ihost_val_prenotify: {'ihost_action': 'unlock'} ihost_val: {'ihost_action': 'unlock', 'task': u'Unlocking'} 2019-01-28 17:20:52.591 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 host.update.ihost_val_prenotify {'ihost_action': 'unlock'} 2019-01-28 17:20:52.591 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 action_check action=unlock, notify_vim=True notify_mtce=True rc=True 2019-01-28 17:20:52.591 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 post action_check hostupdate action=unlock notify_vim=True notify_mtc=True skip_notify_mtce=False 2019-01-28 17:20:52.591 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 stage_action unlock 2019-01-28 17:20:52.592 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 _handle_unlock_action 2019-01-28 17:20:52.592 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 Action staged: unlock 2019-01-28 17:20:52.592 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 post action_stage hostupdate action=unlock notify_vim=True notify_mtc=True skip_notify_mtce=False 2019-01-28 17:20:52.592 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 2. delta_handle ['action'] 2019-01-28 17:20:52.592 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 post delta_handle hostupdate action=unlock notify_vim=True notify_mtc=True skip_notify_mtce=False 2019-01-28 17:20:52.592 21006 INFO sysinv.api.controllers.v1.host [-] update ihost_val_prenotify: {'ihost_action': 'unlock'} 2019-01-28 17:20:52.601 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 apply ihost_val {'ihost_action': 'unlock', 'task': u'Unlocking'} 2019-01-28 17:20:52.601 21006 INFO sysinv.api.controllers.v1.host [-] Notify VIM host action controller-0 action=unlock 2019-01-28 17:20:52.602 21006 WARNING sysinv.api.controllers.v1.vim_api [-] vim_host_action hostname=controller-0, action=unlock 2019-01-28 17:20:52.602 21006 WARNING sysinv.api.controllers.v1.vim_api [-] vim_host_action hostname=controller-0, action=unlock api_cmd=http://localhost:30001/nfvi-plugins/v1/hosts/3a53c151-5f31-42c5-9224-b754ff15d44c headers={'Content-type': 'application/json', 'User-Agent': 'sysinv/1.0'} payload={'action': 'unlock', 'hostname': u'controller-0', 'uuid': u'3a53c151-5f31-42c5-9224-b754ff15d44c'} 2019-01-28 17:20:52.602 21006 INFO sysinv.api.controllers.v1.rest_api [-] PATCH cmd:http://localhost:30001/nfvi-plugins/v1/hosts/3a53c151-5f31-42c5-9224-b754ff15d44c hdr:{'Content-type': 'application/json', 'User-Agent': 'sysinv/1.0'} payload:{"action": "unlock", "hostname": "controller-0", "uuid": "3a53c151-5f31-42c5-9224-b754ff15d44c"} 2019-01-28 17:20:52.606 21006 INFO sysinv.api.controllers.v1.rest_api [-] Response={} 2019-01-28 17:20:52.606 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 Perform configure_ihost. 2019-01-28 17:20:54.587 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 Action unlock perform notify_mtce 2019-01-28 17:20:54.636 21006 INFO sysinv.api.controllers.v1.mtce_api [-] number of calls to rest_api_request=1 (max_retry=3) 2019-01-28 17:20:54.636 21006 INFO sysinv.api.controllers.v1.rest_api [-] PATCH cmd:http://localhost:2112/v1/hosts/3a53c151-5f31-42c5-9224-b754ff15d44c hdr:{'Content-type': 'application/json', 'User-Agent': 'sysinv/1.0'} payload:{"tboot": "false", "ttys_dcd": null, "infra_ip": null, "subfunctions": "controller,worker", "bm_ip": null, "install_state": null, "rootfs_device": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "bm_username": null, "operation": "modify", "serialid": null, "id": 1, "vim_progress_status": null, "console": "tty0", "uuid": "3a53c151-5f31-42c5-9224-b754ff15d44c", "mgmt_ip": "192.168.204.3", "software_load": "19.01", "hostname": "controller-0", "iscsi_initiator_name": "iqn.1994-05.com.redhat:80988ace3499", "capabilities": {"stor_function": "monitor"}, "install_output": "text", "location": {}, "availability": "online", "invprovision": "provisioned", "peer_id": null, "administrative": "locked", "personality": "controller", "recordtype": "standard", "bm_mac": null, "mtce_info": null, "isystem_uuid": "f1ccce70-4b53-4a09-a9fb-803d187a8318", "boot_device": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "install_state_info": null, "mgmt_mac": "00:00:00:00:00:00", "subfunction_oper": "disabled", "target_load": "19.01", "vsc_controllers": null, "operational": "disabled", "subfunction_avail": "online", "action": "unlock", "bm_type": null} 2019-01-28 17:20:54.647 21006 INFO sysinv.api.controllers.v1.rest_api [-] Response={u'status': u'pass'} 2019-01-28 17:20:54.664 21006 INFO sysinv.api.controllers.v1.host [-] host controller-0 ihost_patch_end_2019-01-28-17-20-54 patch 2019-01-28 17:20:54.680 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-20-54 patch 2019-01-28 17:20:54.681 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['subfunction_avail', 'administrative', 'availability'] 2019-01-28 17:20:54.681 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 2. delta_handle ['administrative'] 2019-01-28 17:20:54.681 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 post delta_handle hostupdate action=None notify_vim=False notify_mtc=False skip_notify_mtce=False 2019-01-28 17:20:54.706 21006 INFO sysinv.api.controllers.v1.host [-] host controller-0 ihost_patch_end_2019-01-28-17-20-54 patch 2019-01-28 17:20:54.729 21006 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:21:07.492 20819 INFO oslo_service.service [-] Caught SIGTERM, stopping children 2019-01-28 17:21:07.492 20819 INFO oslo.service.wsgi [-] Stopping WSGI server. 2019-01-28 17:21:07.493 20819 INFO oslo_service.service [-] Waiting on 1 children to exit 2019-01-28 17:21:07.493 21006 INFO oslo.service.wsgi [-] Stopping WSGI server. 2019-01-28 17:21:07.502 20819 INFO oslo_service.service [-] Child 21006 exited with status 0 2019-01-28 17:21:07.504 20819 INFO oslo_service.service [-] Caught SIGTERM, stopping children 2019-01-28 17:21:07.504 20819 INFO oslo.service.wsgi [-] Stopping WSGI server. 2019-01-28 17:21:07.504 20819 INFO oslo_service.service [-] Waiting on 1 children to exit 2019-01-28 17:21:07.505 21008 INFO oslo.service.wsgi [-] Stopping WSGI server. 2019-01-28 17:21:07.507 20819 INFO oslo_service.service [-] Child 21008 exited with status 0 2019-01-28 17:21:09.022 20749 INFO sysinv.openstack.common.service [-] Caught SIGTERM, exiting 2019-01-28 17:21:12.472 32701 ERROR sysinv.openstack.common.rpc.common [-] Failed to consume message from queue: [Errno 104] Connection reset by peer 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common Traceback (most recent call last): 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 564, in ensure 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common return method(*args, **kwargs) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 644, in _consume 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common return connection.drain_events(**kwargs) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 464, in drain_events 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common return self.blocking_read(timeout) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 469, in blocking_read 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common return self.on_inbound_frame(frame) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 68, in on_frame 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common callback(channel, method_sig, buf, None) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 473, in on_inbound_method 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common method_sig, payload, content, 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 142, in dispatch_method 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common listener(*args) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 593, in _on_close 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common self._x_close_ok() 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 608, in _x_close_ok 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common self.send_method(spec.Connection.CloseOk, callback=self._on_close_ok) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 64, in send_method 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common conn.frame_writer(1, self.channel_id, sig, args, content) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 174, in write_frame 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common write(view[:offset]) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 269, in write 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common self._write(s) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 385, in sendall 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common tail = self.send(data, flags) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 379, in send 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common return self._send_loop(self.fd.send, data, flags) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 366, in _send_loop 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common return send_method(data, *args) 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common error: [Errno 104] Connection reset by peer 2019-01-28 17:21:12.472 32701 TRACE sysinv.openstack.common.rpc.common 2019-01-28 17:21:12.474 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:12.479 32701 ERROR sysinv.openstack.common.rpc.common [-] Failed to consume message from queue: [Errno 104] Connection reset by peer 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common Traceback (most recent call last): 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 564, in ensure 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common return method(*args, **kwargs) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/impl_kombu.py", line 644, in _consume 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common return self.connection.drain_events(timeout=timeout) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common return self.transport.drain_events(self.connection, **kwargs) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common return connection.drain_events(**kwargs) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 464, in drain_events 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common return self.blocking_read(timeout) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 469, in blocking_read 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common return self.on_inbound_frame(frame) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 68, in on_frame 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common callback(channel, method_sig, buf, None) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 473, in on_inbound_method 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common method_sig, payload, content, 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 142, in dispatch_method 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common listener(*args) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 593, in _on_close 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common self._x_close_ok() 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 608, in _x_close_ok 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common self.send_method(spec.Connection.CloseOk, callback=self._on_close_ok) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 64, in send_method 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common conn.frame_writer(1, self.channel_id, sig, args, content) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 174, in write_frame 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common write(view[:offset]) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 269, in write 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common self._write(s) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 385, in sendall 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common tail = self.send(data, flags) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 379, in send 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common return self._send_loop(self.fd.send, data, flags) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 366, in _send_loop 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common return send_method(data, *args) 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common error: [Errno 104] Connection reset by peer 2019-01-28 17:21:12.479 32701 TRACE sysinv.openstack.common.rpc.common 2019-01-28 17:21:12.480 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:12.485 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2019-01-28 17:21:12.485 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2019-01-28 17:21:13.486 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:13.490 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:13.494 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds. 2019-01-28 17:21:13.494 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds. 2019-01-28 17:21:16.497 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:16.516 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:16.530 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds. 2019-01-28 17:21:16.531 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds. 2019-01-28 17:21:21.531 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:21.551 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 7 seconds. 2019-01-28 17:21:21.552 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:21.565 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 7 seconds. 2019-01-28 17:21:28.553 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:28.557 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 9 seconds. 2019-01-28 17:21:28.565 32701 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:21:28.570 32701 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 9 seconds. 2019-01-28 17:21:35.666 32701 INFO sysinv.openstack.common.service [-] Caught SIGTERM, exiting 2019-01-28 17:22:23.030 9134 INFO sysinv.agent.lldp.manager [-] Configured sysinv LLDP agent drivers: ['lldpd', 'ovs'] 2019-01-28 17:22:23.081 9134 INFO sysinv.agent.lldp.manager [-] Loaded sysinv LLDP agent drivers: ['lldpd', 'ovs'] 2019-01-28 17:22:23.081 9134 INFO sysinv.agent.lldp.manager [-] Registered sysinv LLDP agent drivers: ['lldpd', 'ovs'] 2019-01-28 17:22:23.127 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2019-01-28 17:22:24.129 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:22:24.133 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds. 2019-01-28 17:22:27.134 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:22:27.139 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds. 2019-01-28 17:22:32.142 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:22:32.146 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 7 seconds. 2019-01-28 17:22:39.148 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:22:39.153 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 9 seconds. 2019-01-28 17:22:48.157 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:22:48.164 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 11 seconds. 2019-01-28 17:22:59.170 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:22:59.174 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 13 seconds. 2019-01-28 17:23:12.181 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:23:12.186 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 15 seconds. 2019-01-28 17:23:27.187 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:23:27.190 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 17 seconds. 2019-01-28 17:23:44.192 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:23:44.196 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 19 seconds. 2019-01-28 17:24:03.201 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:24:03.205 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 21 seconds. 2019-01-28 17:24:24.213 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:24:24.217 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 23 seconds. 2019-01-28 17:24:47.236 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:24:47.240 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 25 seconds. 2019-01-28 17:24:57.238 30107 INFO manage-partitions [-] Called partition 'check' with 'None' 'None' and '[{"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part1", "start_mib": 1, "uuid": "491211cd-f682-41ca-bc32-67f019b7b95b", "device_node": "/dev/sda1", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "21686148-6449-6e6f-744e-656564454649", "size_mib": 1}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part2", "start_mib": 2, "uuid": "24932520-86e1-4917-a18f-f5b26d88040c", "device_node": "/dev/sda2", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 500}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part3", "start_mib": 502, "uuid": "e88cae09-e541-4edb-8bed-235d73800070", "device_node": "/dev/sda3", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 20000}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part4", "start_mib": 20502, "uuid": "40be4c0d-7110-434b-819d-cbb92db19b04", "device_node": "/dev/sda4", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "e6d6d379-f507-44c2-a23c-238f2a3df928", "size_mib": 224256}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5", "start_mib": 244758, "uuid": "c0c46c7d-bae2-44c7-95d5-fd832c229500", "device_node": "/dev/sda5", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ba5eba11-0000-1111-2222-000000000001", "size_mib": 6144}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part6", "start_mib": 250902, "uuid": "d9826c60-d26c-4e3e-9b58-7dc31af211dd", "device_node": "/dev/sda6", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ba5eba11-0000-1111-2222-000000000001", "size_mib": 34816}]' 2019-01-28 17:24:57.239 30107 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:57.241 30107 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:24:57.463 30107 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:57.467 30107 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:24:57.693 30107 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:57.701 30107 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:24:57.951 30107 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:57.960 30107 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:24:58.208 30107 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:58.217 30107 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:24:58.461 30107 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:58.469 30107 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:24:58.712 30107 INFO manage-partitions [-] Executing command: 'sgdisk -i 1 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:58.741 30107 INFO manage-partitions [-] Executing command: 'sgdisk -i 2 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:58.756 30107 INFO manage-partitions [-] Executing command: 'sgdisk -i 3 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:58.770 30107 INFO manage-partitions [-] Executing command: 'sgdisk -i 4 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:58.778 30107 INFO manage-partitions [-] Executing command: 'sgdisk -i 5 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:24:58.785 30107 INFO manage-partitions [-] Executing command: 'sgdisk -i 6 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:25:12.243 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:25:12.261 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 27 seconds. 2019-01-28 17:25:39.274 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:25:39.293 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 29 seconds. 2019-01-28 17:26:08.295 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:26:08.312 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2019-01-28 17:26:38.313 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:26:38.331 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2019-01-28 17:27:08.334 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:27:08.352 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2019-01-28 17:27:38.370 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:27:38.387 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2019-01-28 17:28:08.405 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:28:08.417 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2019-01-28 17:28:38.434 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:28:38.437 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2019-01-28 17:29:08.463 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:29:08.467 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2019-01-28 17:29:38.486 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:29:38.490 9134 ERROR sysinv.openstack.common.rpc.common [-] AMQP server on 192.168.204.2:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2019-01-28 17:29:46.698 11979 INFO keystonemiddleware.auth_token [-] Starting Keystone auth_token middleware 2019-01-28 17:29:46.700 11979 WARNING keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. 2019-01-28 17:29:46.703 11979 INFO oslo.service.wsgi [-] sysinv_api listening on 192.168.204.2:6385 2019-01-28 17:29:46.704 11979 INFO oslo_service.service [-] Starting 1 workers 2019-01-28 17:29:46.707 11979 INFO keystonemiddleware.auth_token [-] Starting Keystone auth_token middleware 2019-01-28 17:29:46.710 11979 WARNING keystonemiddleware._common.config [-] The option "user_domain_name" is not known to keystonemiddleware 2019-01-28 17:29:46.710 11979 WARNING keystonemiddleware._common.config [-] The option "auth_url" is not known to keystonemiddleware 2019-01-28 17:29:46.710 11979 WARNING keystonemiddleware._common.config [-] The option "project_domain_name" is not known to keystonemiddleware 2019-01-28 17:29:46.710 11979 WARNING keystonemiddleware._common.config [-] The option "username" is not known to keystonemiddleware 2019-01-28 17:29:46.710 11979 WARNING keystonemiddleware._common.config [-] The option "project_name" is not known to keystonemiddleware 2019-01-28 17:29:46.710 11979 WARNING keystonemiddleware._common.config [-] The option "password" is not known to keystonemiddleware 2019-01-28 17:29:46.710 11979 WARNING keystonemiddleware.auth_token [-] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. 2019-01-28 17:29:46.713 11979 INFO oslo.service.wsgi [-] sysinv_api_pxe listening on 169.254.202.2:6385 2019-01-28 17:29:46.713 11979 INFO oslo_service.service [-] Starting 1 workers 2019-01-28 17:29:47.134 11851 INFO sysinv.conductor.manager [-] sysinv-conductor start committed system={'system_mode': u'simplex', 'created_at': datetime.datetime(2019, 1, 28, 16, 48, 14, 669370, tzinfo=), 'uuid': u'f1ccce70-4b53-4a09-a9fb-803d187a8318', 'software_version': u'19.01', 'service_project_name': u'services', 'system_type': u'All-in-one', 'name': u'06d5c304-20d0-4d34-b76a-42036df314dc', 'description': None, 'location': None, 'updated_at': datetime.datetime(2019, 1, 28, 16, 49, 0, 660043, tzinfo=), 'capabilities': {u'shared_services': u'[]', u'sdn_enabled': False, u'vswitch_type': u'ovs-dpdk', u'https_enabled': False, u'kubernetes_enabled': True, u'region_config': False}, 'id': 1, 'contact': None, 'security_feature': u'nopti nospectre_v2', 'services': 72, 'timezone': u'UTC', 'security_profile': u'standard', 'distributed_cloud_role': None, 'region_name': u'RegionOne'} 2019-01-28 17:29:47.191 11851 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:29:49.461 12613 WARNING keystonemiddleware.auth_token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. 2019-01-28 17:30:01.850 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:30:01.974 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-30-01 patch 2019-01-28 17:30:01.974 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_end. No changes from mtce/1.0. 2019-01-28 17:30:06.754 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-30-06 patch 2019-01-28 17:30:06.754 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['availability'] 2019-01-28 17:30:06.791 12613 INFO sysinv.api.controllers.v1.host [-] host controller-0 ihost_patch_end_2019-01-28-17-30-06 patch 2019-01-28 17:30:06.843 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:30:08.509 9134 INFO sysinv.openstack.common.rpc.common [-] Reconnecting to AMQP server on 192.168.204.2:5672 2019-01-28 17:30:08.515 9134 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:30:08.523 9134 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report. 2019-01-28 17:30:09.050 9134 WARNING sysinv.agent.pci [-] Enabling device ens4 to query link speed 2019-01-28 17:30:09.053 9134 WARNING sysinv.agent.pci [-] ATTR speed unknown for: ens4 (flags: 0x1002) 2019-01-28 17:30:09.053 9134 WARNING sysinv.agent.pci [-] Disabling device ens4 after querying link speed 2019-01-28 17:30:09.309 9134 WARNING sysinv.agent.pci [-] Enabling device eth1000 to query link speed 2019-01-28 17:30:09.311 9134 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1000 (flags: 0x1002) 2019-01-28 17:30:09.312 9134 WARNING sysinv.agent.pci [-] Disabling device eth1000 after querying link speed 2019-01-28 17:30:09.558 9134 WARNING sysinv.agent.pci [-] Enabling device eth1001 to query link speed 2019-01-28 17:30:09.561 9134 WARNING sysinv.agent.pci [-] ATTR speed unknown for: eth1001 (flags: 0x1002) 2019-01-28 17:30:09.561 9134 WARNING sysinv.agent.pci [-] Disabling device eth1001 after querying link speed 2019-01-28 17:30:09.581 9134 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:30:09.604 11851 INFO sysinv.openstack.common.rpc.common [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:30:09.642 9134 INFO sysinv.agent.manager [-] Sysinv Agent platform update by host: {} 2019-01-28 17:30:09.642 9134 INFO sysinv.agent.manager [-] Agent found matching ihost: 3a53c151-5f31-42c5-9224-b754ff15d44c 2019-01-28 17:30:10.056 11851 ERROR sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] Failed in neutron stuff 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager Traceback (most recent call last): 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 4385, in subfunctions_update_by_ihost 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager context, ihost['hostname']) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/openstack.py", line 247, in get_neutron_host_id_by_name 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager hosts = client.list_hosts() 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1448, in list_hosts 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager return self.get(self.hosts_path, params=_params) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 355, in get 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 332, in retry_request 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 283, in do_request 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager resp, replybody = self.httpclient.do_request(action, method, body=body) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 344, in do_request 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager return self.request(url, method, **kwargs) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 332, in request 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager resp = super(SessionClient, self).request(*args, **kwargs) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 192, in request 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager return self.session.request(url, method, **kwargs) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager return wrapped(*args, **kwargs) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 430, in request 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager resp = send(**kwargs) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 474, in _send_request 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager raise exceptions.ConnectionRefused(msg) 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager ConnectFailure: Unable to establish connection to http://192.168.204.2:9696/v2.0/hosts 2019-01-28 17:30:10.056 11851 TRACE sysinv.conductor.manager 2019-01-28 17:30:10.248 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] port 6f08deda-c34f-40d6-9c24-f39a49e62236 update attr: {'sriov_numvfs': 0, 'driver': u'e1000', 'sriov_vfs_pci_address': u'', 'sriov_totalvfs': None, 'dpdksupport': False, 'speed': u'1000'} 2019-01-28 17:30:10.422 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] port c6876c8f-09d6-4068-960a-d7bd546de3b2 update attr: {'sriov_numvfs': 0, 'driver': u'e1000', 'sriov_vfs_pci_address': u'', 'sriov_totalvfs': None, 'dpdksupport': False, 'speed': None} 2019-01-28 17:30:10.509 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] port 2e944997-9b41-4d13-b3e6-2670d373d1e1 update attr: {'sriov_numvfs': 0, 'driver': u'virtio-pci', 'sriov_vfs_pci_address': u'', 'sriov_totalvfs': None, 'dpdksupport': True, 'speed': None} 2019-01-28 17:30:10.595 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] port 0b435963-e0f6-47cc-a642-cddae4657b74 update attr: {'sriov_numvfs': 0, 'driver': u'virtio-pci', 'sriov_vfs_pci_address': u'', 'sriov_totalvfs': None, 'dpdksupport': True, 'speed': None} 2019-01-28 17:30:10.637 11851 ERROR sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] Failed in neutron stuff 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager Traceback (most recent call last): 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py", line 4385, in subfunctions_update_by_ihost 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager context, ihost['hostname']) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib64/python2.7/site-packages/sysinv/conductor/openstack.py", line 247, in get_neutron_host_id_by_name 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager hosts = client.list_hosts() 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1448, in list_hosts 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager return self.get(self.hosts_path, params=_params) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 355, in get 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 332, in retry_request 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager headers=headers, params=params) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 283, in do_request 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager resp, replybody = self.httpclient.do_request(action, method, body=body) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 344, in do_request 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager return self.request(url, method, **kwargs) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 332, in request 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager resp = super(SessionClient, self).request(*args, **kwargs) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 192, in request 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager return self.session.request(url, method, **kwargs) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager return wrapped(*args, **kwargs) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 430, in request 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager resp = send(**kwargs) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 474, in _send_request 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager raise exceptions.ConnectionRefused(msg) 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager ConnectFailure: Unable to establish connection to http://192.168.204.2:9696/v2.0/hosts 2019-01-28 17:30:10.637 11851 TRACE sysinv.conductor.manager 2019-01-28 17:30:10.685 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] attr: {'sriov_numvfs': 0, 'driver': u'ata_piix', 'psvendor': u'Red Hat, Inc.', 'pdevice_id': u'7010', 'pclass': u'IDE interface', 'psdevice': u'Qemu virtual machine', 'sriov_vfs_pci_address': u'', 'pvendor': u'Intel Corporation', 'pvendor_id': u'8086', 'pclass_id': u'010180', 'sriov_totalvfs': None} 2019-01-28 17:30:10.702 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] attr: {'sriov_numvfs': 0, 'driver': u'uhci_hcd', 'psvendor': u'Red Hat, Inc.', 'pdevice_id': u'7020', 'pclass': u'USB controller', 'psdevice': u'QEMU Virtual Machine', 'sriov_vfs_pci_address': u'', 'pvendor': u'Intel Corporation', 'pvendor_id': u'8086', 'pclass_id': u'0c0300', 'sriov_totalvfs': None} 2019-01-28 17:30:10.717 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] attr: {'sriov_numvfs': 0, 'driver': None, 'psvendor': u'QEMU Virtual Machine', 'pdevice_id': u'00b8', 'pclass': u'VGA compatible controller', 'psdevice': u'QEMU Virtual Machine', 'sriov_vfs_pci_address': u'', 'pvendor': u'Cirrus Logic', 'pvendor_id': u'1013', 'pclass_id': u'030000', 'sriov_totalvfs': None} 2019-01-28 17:30:10.731 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] attr: {'sriov_numvfs': 0, 'driver': u'ahci', 'psvendor': u'-p01', 'pdevice_id': u'2922', 'pclass': u'SATA controller', 'psdevice': u'Red Hat, Inc.', 'sriov_vfs_pci_address': u'', 'pvendor': u'Intel Corporation', 'pvendor_id': u'8086', 'pclass_id': u'010601', 'sriov_totalvfs': None} 2019-01-28 17:30:10.748 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] attr: {'sriov_numvfs': 0, 'driver': u'virtio-pci', 'psvendor': u'Red Hat, Inc.', 'pdevice_id': u'1002', 'pclass': u'Unclassified device [00ff]', 'psdevice': u'Device 0005', 'sriov_vfs_pci_address': u'', 'pvendor': u'Red Hat, Inc.', 'pvendor_id': u'1af4', 'pclass_id': u'00ff00', 'sriov_totalvfs': None} 2019-01-28 17:30:10.803 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] Already in db numa_node=0 mynuma_nodes=[0] 2019-01-28 17:30:10.845 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] Logical CPU topology: host:controller-0 (controller,worker), sockets:1, cores/socket=6, threads/core=1, reference:current (unchanged) 2019-01-28 17:30:10.845 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] cpu_id : 0 1 2 3 4 5 2019-01-28 17:30:10.845 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] socket_id : 0 0 0 0 0 0 2019-01-28 17:30:10.845 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] core_id : 0 1 2 3 4 5 2019-01-28 17:30:10.846 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] thread_id : 0 0 0 0 0 0 2019-01-28 17:30:12.007 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:30:12.105 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:30:12.575 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] Partition create on host: 1. Details: {'status': 1, 'forihostid': 1, u'device_path': u'/dev/disk/by-path/pci-0000:00:07.0-ata-2.0-part2', u'start_mib': u'1', u'uuid': u'ba7db0e3-88e2-4280-aca3-30f6cb4cf659', 'idisk_id': 2, u'type_name': u'ceph journal', u'device_node': u'/dev/sdb2', u'end_mib': u'1025', 'idisk_uuid': u'd26bee13-ab6f-43fd-8a3e-77e69d75e846', u'type_guid': u'45b0969e-9b03-4f30-b4c6-b4b80ceff106', u'size_mib': u'1024'} 2019-01-28 17:30:12.599 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] Partition create on host: 1. Details: {'status': 1, 'forihostid': 1, u'device_path': u'/dev/disk/by-path/pci-0000:00:07.0-ata-2.0-part1', u'start_mib': u'1025', u'uuid': u'b9a0bb8d-6c8a-4459-9fca-1f6e8f7275db', 'idisk_id': 2, u'type_name': u'ceph data', u'device_node': u'/dev/sdb1', u'end_mib': u'256000', 'idisk_uuid': u'd26bee13-ab6f-43fd-8a3e-77e69d75e846', u'type_guid': u'4fbd7e29-9d25-41b8-afd0-062c0ceff05d', u'size_mib': u'254975'} 2019-01-28 17:30:12.791 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:30:12.902 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] platform_interfaces host_id=1 info_list=[] 2019-01-28 17:30:12.923 9134 INFO sysinv.agent.manager [-] _wait_for_nova_lvg: Must wait before reporting node availability. Conductor sees unprovisioned nova-local state. Would result in an invalid host aggregate assignment. 2019-01-28 17:30:12.937 9134 INFO sysinv.agent.manager [-] _wait_for_nova_lvg: Must wait before reporting node availability. Conductor sees unprovisioned nova-local state. Would result in an invalid host aggregate assignment. 2019-01-28 17:30:14.613 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] PV not found in Agent. uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f current state: adding 2019-01-28 17:30:14.686 9134 INFO sysinv.agent.manager [-] Agent config applied daeec83f-c5a4-4c66-b919-b01ee23c6bcb 2019-01-28 17:30:14.718 11851 INFO sysinv.conductor.manager [req-cb90af22-33aa-495f-b7a2-d8370f8bbfe5 None None] SYS_I Clear system config alarm: controller-0 2019-01-28 17:30:16.061 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:30:16.150 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-30-16 patch 2019-01-28 17:30:16.150 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['availability', 'operational'] 2019-01-28 17:30:16.245 12613 INFO sysinv.api.controllers.v1.host [-] host controller-0 ihost_patch_end_2019-01-28-17-30-16 patch 2019-01-28 17:30:16.293 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:30:21.352 2574 INFO manage-partitions [-] Called partition 'check' with 'None' 'None' and '[{"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part1", "start_mib": 1, "uuid": "491211cd-f682-41ca-bc32-67f019b7b95b", "device_node": "/dev/sda1", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "21686148-6449-6e6f-744e-656564454649", "size_mib": 1}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part2", "start_mib": 2, "uuid": "24932520-86e1-4917-a18f-f5b26d88040c", "device_node": "/dev/sda2", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 500}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part3", "start_mib": 502, "uuid": "e88cae09-e541-4edb-8bed-235d73800070", "device_node": "/dev/sda3", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7", "size_mib": 20000}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part4", "start_mib": 20502, "uuid": "40be4c0d-7110-434b-819d-cbb92db19b04", "device_node": "/dev/sda4", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "e6d6d379-f507-44c2-a23c-238f2a3df928", "size_mib": 224256}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part5", "start_mib": 244758, "uuid": "c0c46c7d-bae2-44c7-95d5-fd832c229500", "device_node": "/dev/sda5", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ba5eba11-0000-1111-2222-000000000001", "size_mib": 6144}, {"device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0-part6", "start_mib": 250902, "uuid": "d9826c60-d26c-4e3e-9b58-7dc31af211dd", "device_node": "/dev/sda6", "disk_device_path": "/dev/disk/by-path/pci-0000:00:07.0-ata-1.0", "type_guid": "ba5eba11-0000-1111-2222-000000000001", "size_mib": 34816}]' 2019-01-28 17:30:21.353 2574 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:21.355 2574 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:30:21.573 2574 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:21.589 2574 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:30:21.850 2574 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:21.853 2574 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:30:22.073 2574 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:22.090 2574 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:30:22.340 2574 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:22.350 2574 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:30:22.637 2574 INFO manage-partitions [-] Executing command: 'udevadm settle -E /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:22.647 2574 INFO manage-partitions [-] Executing command: 'parted -s /dev/disk/by-path/pci-0000:00:07.0-ata-1.0 print' 2019-01-28 17:30:22.916 2574 INFO manage-partitions [-] Executing command: 'sgdisk -i 1 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:22.931 2574 INFO manage-partitions [-] Executing command: 'sgdisk -i 2 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:22.950 2574 INFO manage-partitions [-] Executing command: 'sgdisk -i 3 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:22.980 2574 INFO manage-partitions [-] Executing command: 'sgdisk -i 4 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:22.998 2574 INFO manage-partitions [-] Executing command: 'sgdisk -i 5 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:23.020 2574 INFO manage-partitions [-] Executing command: 'sgdisk -i 6 /dev/disk/by-path/pci-0000:00:07.0-ata-1.0' 2019-01-28 17:30:47.329 11851 INFO sysinv.conductor.manager [-] Sending agent update request for host 1 to update (audit_lvg, audit_pv) 2019-01-28 17:30:47.335 11851 INFO sysinv.agent.rpcapi [-] AgentApi.agent_update: fanout_cast: sending update request to agent for: (audit_lvg, audit_pv) 2019-01-28 17:30:47.369 9134 INFO sysinv.agent.manager [req-d4f91804-e0df-4e95-a93c-fcfa6e1babc1 admin None] _wait_for_nova_lvg: Must wait before reporting node availability. Conductor sees unprovisioned nova-local state. Would result in an invalid host aggregate assignment. 2019-01-28 17:30:47.400 9134 INFO sysinv.agent.manager [req-d4f91804-e0df-4e95-a93c-fcfa6e1babc1 admin None] _wait_for_nova_lvg: Must wait before reporting node availability. Conductor sees unprovisioned nova-local state. Would result in an invalid host aggregate assignment. 2019-01-28 17:30:48.946 11851 INFO sysinv.conductor.manager [req-3b190b81-690b-42cc-8051-617a2a09d477 None None] PV uuid: ebb29181-52b4-42aa-bef2-054c61c22c6f changed UUID from None to tCsJEM-uj3u-sdee-z1nY-svMc-K1oB-E1LUIb 2019-01-28 17:30:49.126 11851 INFO sysinv.conductor.manager [req-3b190b81-690b-42cc-8051-617a2a09d477 None None] LVG uuid: 44ca1c2a-8dd6-42b0-b04c-32940fbb802d changed UUID from None to qNTI7X-IUju-Jfcv-WFyZ-us7n-QbyU-O7UeWf 2019-01-28 17:30:50.082 11851 INFO sysinv.conductor.manager [-] _controller_config_active_apply about to resize the filesystem 2019-01-28 17:30:50.082 11851 WARNING sysinv.conductor.manager [-] resizing filesystems 2019-01-28 17:30:50.082 9134 INFO sysinv.agent.manager [req-d4f91804-e0df-4e95-a93c-fcfa6e1babc1 admin None] config_apply_runtime_manifest: daeec83f-c5a4-4c66-b919-b01ee23c6bcb {u'classes': [u'openstack::keystone::endpoint::runtime'], u'force': False, u'personalities': [u'controller'], u'host_uuids': u'3a53c151-5f31-42c5-9224-b754ff15d44c'} controller 2019-01-28 17:30:50.083 9134 INFO sysinv.agent.manager [req-d4f91804-e0df-4e95-a93c-fcfa6e1babc1 admin None] controller-active 2019-01-28 17:30:50.083 9134 INFO sysinv.agent.manager [req-d4f91804-e0df-4e95-a93c-fcfa6e1babc1 admin None] _apply_runtime_manifest with hieradata_path = '/opt/platform/puppet/19.01/hieradata' 2019-01-28 17:30:50.138 11851 INFO sysinv.conductor.manager [-] drbd-overview: pgsql-40.0, cgcs-9.9, extension-0.96875, patch-vault-0, etcd-4.8, dockerdistribution-16.0 2019-01-28 17:30:50.139 11851 INFO sysinv.conductor.manager [-] lvdisplay: pgsql-40.0, cgcs-10.0, extension-1.0, patch-vault-0, etcd-5.0, dockerdistribution-16.0 2019-01-28 17:30:50.147 11851 INFO sysinv.conductor.manager [-] SYS_I Clear system config alarm: controller-0 2019-01-28 17:30:54.541 12613 INFO sysinv.api.controllers.v1.label [-] patch_data: {'openstack-control-plane': 'enabled'} 2019-01-28 17:30:54.593 12613 INFO sysinv.openstack.common.rpc.common [-] Connected to AMQP server on 192.168.204.2:5672 2019-01-28 17:30:54.597 11851 INFO sysinv.conductor.manager [req-ee091d2f-95a0-4f93-a57c-ee4a83e7efe4 admin admin] update_kubernetes_label: label_dict={u'openstack-control-plane': u'enabled'} 2019-01-28 17:30:54.647 12613 INFO sysinv.api.controllers.v1.rest_api [-] PATCH cmd:http://localhost:30001/nfvi-plugins/v1/hosts hdr:{'Content-type': 'application/json', 'User-Agent': 'sysinv/1.0'} payload:{"hostname": "controller-0", "uuid": "3a53c151-5f31-42c5-9224-b754ff15d44c"} 2019-01-28 17:30:54.649 12613 INFO sysinv.api.controllers.v1.rest_api [-] Response={u'status': u'success'} 2019-01-28 17:31:07.271 12613 INFO sysinv.api.controllers.v1.label [-] patch_data: {'openstack-compute-node': 'enabled'} 2019-01-28 17:31:07.284 11851 INFO sysinv.conductor.manager [req-38e81a8a-3f4c-4d25-84ec-8fff0bc40260 admin admin] update_kubernetes_label: label_dict={u'openstack-compute-node': u'enabled'} 2019-01-28 17:31:07.321 12613 INFO sysinv.api.controllers.v1.rest_api [-] PATCH cmd:http://localhost:30001/nfvi-plugins/v1/hosts hdr:{'Content-type': 'application/json', 'User-Agent': 'sysinv/1.0'} payload:{"hostname": "controller-0", "uuid": "3a53c151-5f31-42c5-9224-b754ff15d44c"} 2019-01-28 17:31:07.323 12613 INFO sysinv.api.controllers.v1.rest_api [-] Response={u'status': u'success'} 2019-01-28 17:31:15.346 12613 INFO sysinv.api.controllers.v1.label [-] patch_data: {'openvswitch': 'enabled'} 2019-01-28 17:31:15.357 11851 INFO sysinv.conductor.manager [req-2b180e8b-766b-40d1-9c88-170a2e22e9c4 admin admin] update_kubernetes_label: label_dict={u'openvswitch': u'enabled'} 2019-01-28 17:31:15.423 12613 INFO sysinv.api.controllers.v1.rest_api [-] PATCH cmd:http://localhost:30001/nfvi-plugins/v1/hosts hdr:{'Content-type': 'application/json', 'User-Agent': 'sysinv/1.0'} payload:{"hostname": "controller-0", "uuid": "3a53c151-5f31-42c5-9224-b754ff15d44c"} 2019-01-28 17:31:15.424 12613 INFO sysinv.api.controllers.v1.rest_api [-] Response={u'status': u'success'} 2019-01-28 17:31:21.055 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['uptime', 'task'] 2019-01-28 17:31:21.159 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-31-21 patch 2019-01-28 17:31:21.159 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_end. No changes from mtce/1.0. 2019-01-28 17:31:21.208 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-31-21 patch 2019-01-28 17:31:21.208 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['subfunction_oper', 'subfunction_avail'] 2019-01-28 17:31:21.266 12613 INFO sysinv.api.controllers.v1.host [-] host controller-0 ihost_patch_end_2019-01-28-17-31-21 patch 2019-01-28 17:31:21.589 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-31-21 patch 2019-01-28 17:31:21.590 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 1. delta_handle ['action'] 2019-01-28 17:31:21.590 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 Pending update_vim_progress_status services-enabled 2019-01-28 17:31:21.590 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 action=services-enabled ihost_val_prenotify: {} ihost_val: {} 2019-01-28 17:31:21.590 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 action_check action=services-enabled, notify_vim=False notify_mtce=False rc=True 2019-01-28 17:31:21.590 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 post action_check hostupdate action=services-enabled notify_vim=False notify_mtc=False skip_notify_mtce=False 2019-01-28 17:31:21.590 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 stage_action services-enabled 2019-01-28 17:31:21.590 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 received services-enabled task= vim_progress_status= 2019-01-28 17:31:21.591 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 Action staged: services-enabled 2019-01-28 17:31:21.591 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 post action_stage hostupdate action=services-enabled notify_vim=False notify_mtc=False skip_notify_mtce=True 2019-01-28 17:31:21.591 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 2. delta_handle ['action'] 2019-01-28 17:31:21.591 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 post delta_handle hostupdate action=services-enabled notify_vim=False notify_mtc=False skip_notify_mtce=True 2019-01-28 17:31:21.591 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 apply ihost_val {'vim_progress_status': 'services-enabled'} 2019-01-28 17:31:21.644 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 notify_availability=services-enabled 2019-01-28 17:31:21.718 12613 INFO sysinv.api.controllers.v1.host [-] host controller-0 ihost_patch_end_2019-01-28-17-31-21 patch 2019-01-28 17:32:46.440 9134 INFO sysinv.agent.manager [-] iscsi initiator name = iqn.1994-05.com.redhat:80988ace3499 2019-01-28 17:32:47.121 11851 INFO sysinv.conductor.manager [req-85b87895-a2cf-40d7-aeb8-8835cafa6ff5 None None] SYS_I Clear system config alarm: controller-0 2019-01-28 17:32:47.178 9134 INFO sysinv.agent.manager [-] Sysinv Agent platform update by host: {'config_applied': 'daeec83f-c5a4-4c66-b919-b01ee23c6bcb', 'availability': 'available', 'iscsi_initiator_name': 'iqn.1994-05.com.redhat:80988ace3499'} 2019-01-28 17:33:45.588 12613 INFO sysinv.common.utils [-] Checksum file is included and validated. 2019-01-28 17:33:46.130 12613 INFO sysinv.api.controllers.v1.kube_app [-] Tar file of application stx-openstack verified. 2019-01-28 17:33:46.152 11851 INFO sysinv.conductor.kube_app [-] Application (stx-openstack) upload started. 2019-01-28 17:33:46.723 11851 INFO sysinv.conductor.kube_app [-] Starting Armada service... 2019-01-28 17:33:46.724 11851 INFO sysinv.conductor.kube_app [-] kube_config=/opt/platform/armada/19.01/admin.conf, manifests_dir=/opt/platform/armada/19.01, overrides_dir=/opt/platform/helm/19.01. 2019-01-28 17:33:47.004 11851 INFO sysinv.conductor.kube_app [-] Armada service started! 2019-01-28 17:33:48.648 11851 INFO sysinv.conductor.kube_app [-] Manifest file /manifests/stx-openstack-manifest-no-tests.yaml was successfully validated. 2019-01-28 17:33:48.759 11851 INFO sysinv.conductor.kube_app [-] Helm chart magnum validated 2019-01-28 17:33:48.907 11851 INFO sysinv.conductor.kube_app [-] Helm chart aodh validated 2019-01-28 17:33:48.966 11851 INFO sysinv.conductor.kube_app [-] Helm chart ingress validated 2019-01-28 17:33:49.107 11851 INFO sysinv.conductor.kube_app [-] Helm chart gnocchi validated 2019-01-28 17:33:49.183 11851 INFO sysinv.conductor.kube_app [-] Helm chart barbican validated 2019-01-28 17:33:49.365 11851 INFO sysinv.conductor.kube_app [-] Helm chart keystone validated 2019-01-28 17:33:49.419 11851 INFO sysinv.conductor.kube_app [-] Helm chart openvswitch validated 2019-01-28 17:33:49.531 11851 INFO sysinv.conductor.kube_app [-] Helm chart glance validated 2019-01-28 17:33:49.584 11851 INFO sysinv.conductor.kube_app [-] Helm chart memcached validated 2019-01-28 17:33:49.727 11851 INFO sysinv.conductor.kube_app [-] Helm chart neutron validated 2019-01-28 17:33:49.788 11851 INFO sysinv.conductor.kube_app [-] Helm chart nova-api-proxy validated 2019-01-28 17:33:49.895 11851 INFO sysinv.conductor.kube_app [-] Helm chart cinder validated 2019-01-28 17:33:49.989 11851 INFO sysinv.conductor.kube_app [-] Helm chart panko validated 2019-01-28 17:33:50.041 11851 INFO sysinv.conductor.kube_app [-] Helm chart rbd-provisioner validated 2019-01-28 17:33:50.602 11851 INFO sysinv.conductor.kube_app [-] Helm chart nova validated 2019-01-28 17:33:50.752 11851 INFO sysinv.conductor.kube_app [-] Helm chart horizon validated 2019-01-28 17:33:50.851 11851 INFO sysinv.conductor.kube_app [-] Helm chart heat validated 2019-01-28 17:33:50.905 11851 INFO sysinv.conductor.kube_app [-] Helm chart libvirt validated 2019-01-28 17:33:50.984 11851 INFO sysinv.conductor.kube_app [-] Helm chart mariadb validated 2019-01-28 17:33:51.125 11851 INFO sysinv.conductor.kube_app [-] Helm chart ironic validated 2019-01-28 17:33:51.184 11851 INFO sysinv.conductor.kube_app [-] Helm chart rabbitmq validated 2019-01-28 17:33:51.474 11851 INFO sysinv.conductor.kube_app [-] Helm chart ceilometer validated 2019-01-28 17:33:51.551 11851 INFO sysinv.conductor.kube_app [-] Helm chart rabbitmq-0.1.0.tgz uploaded 2019-01-28 17:33:51.606 11851 INFO sysinv.conductor.kube_app [-] Helm chart neutron-0.1.0.tgz uploaded 2019-01-28 17:33:51.662 11851 INFO sysinv.conductor.kube_app [-] Helm chart mariadb-0.1.0.tgz uploaded 2019-01-28 17:33:51.722 11851 INFO sysinv.conductor.kube_app [-] Helm chart keystone-0.1.0.tgz uploaded 2019-01-28 17:33:51.789 11851 INFO sysinv.conductor.kube_app [-] Helm chart panko-0.1.0.tgz uploaded 2019-01-28 17:33:51.855 11851 INFO sysinv.conductor.kube_app [-] Helm chart openvswitch-0.1.0.tgz uploaded 2019-01-28 17:33:51.923 11851 INFO sysinv.conductor.kube_app [-] Helm chart ironic-0.1.0.tgz uploaded 2019-01-28 17:33:51.996 11851 INFO sysinv.conductor.kube_app [-] Helm chart rbd-provisioner-0.1.0.tgz uploaded 2019-01-28 17:33:52.070 11851 INFO sysinv.conductor.kube_app [-] Helm chart nova-0.1.0.tgz uploaded 2019-01-28 17:33:52.147 11851 INFO sysinv.conductor.kube_app [-] Helm chart gnocchi-0.1.0.tgz uploaded 2019-01-28 17:33:52.234 11851 INFO sysinv.conductor.kube_app [-] Helm chart horizon-0.1.0.tgz uploaded 2019-01-28 17:33:52.309 11851 INFO sysinv.conductor.kube_app [-] Helm chart memcached-0.1.0.tgz uploaded 2019-01-28 17:33:52.387 11851 INFO sysinv.conductor.kube_app [-] Helm chart magnum-0.1.0.tgz uploaded 2019-01-28 17:33:52.468 11851 INFO sysinv.conductor.kube_app [-] Helm chart ingress-0.1.0.tgz uploaded 2019-01-28 17:33:52.550 11851 INFO sysinv.conductor.kube_app [-] Helm chart ceilometer-0.1.0.tgz uploaded 2019-01-28 17:33:52.639 11851 INFO sysinv.conductor.kube_app [-] Helm chart glance-0.1.0.tgz uploaded 2019-01-28 17:33:52.724 11851 INFO sysinv.conductor.kube_app [-] Helm chart libvirt-0.1.0.tgz uploaded 2019-01-28 17:33:52.816 11851 INFO sysinv.conductor.kube_app [-] Helm chart helm-toolkit-0.1.0.tgz uploaded 2019-01-28 17:33:52.910 11851 INFO sysinv.conductor.kube_app [-] Helm chart barbican-0.1.0.tgz uploaded 2019-01-28 17:33:53.003 11851 INFO sysinv.conductor.kube_app [-] Helm chart aodh-0.1.0.tgz uploaded 2019-01-28 17:33:53.114 11851 INFO sysinv.conductor.kube_app [-] Helm chart heat-0.1.0.tgz uploaded 2019-01-28 17:33:53.211 11851 INFO sysinv.conductor.kube_app [-] Helm chart cinder-0.1.0.tgz uploaded 2019-01-28 17:33:53.315 11851 INFO sysinv.conductor.kube_app [-] Helm chart nova-api-proxy-0.1.0.tgz uploaded 2019-01-28 17:33:53.315 11851 INFO sysinv.conductor.kube_app [-] Generating application overrides... 2019-01-28 17:33:53.687 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_neutron_ml2_config={'ml2': {'type_drivers': 'managed_flat,managed_vlan,managed_vxlan', 'tenant_network_types': 'vlan,vxlan', 'mechanism_drivers': 'openvswitch,sriovnicswitch,l2population', 'path_mtu': 0, 'physical_network_mtus': 'physnet0:1500,physnet1:1500'}, 'securitygroup': {'firewall_driver': 'noop'}} 2019-01-28 17:33:53.756 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:33:53.772 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:33:53.988 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:33:54.005 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:33:54.133 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:33:54.148 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:33:54.272 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:33:54.288 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:33:54.411 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:33:54.428 11851 INFO sysinv.helm.neutron [req-5d8b4f74-bd8c-4865-9bf2-e5a3a1f20193 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:33:55.487 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/kube-system-ingress.yaml has no img tags 2019-01-28 17:33:55.489 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/kube-system-ingress.yaml updated with new image tags 2019-01-28 17:33:55.510 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-ingress.yaml has no img tags 2019-01-28 17:33:55.511 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-ingress.yaml updated with new image tags 2019-01-28 17:33:55.523 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-rbd-provisioner.yaml updated with new image tags 2019-01-28 17:33:55.565 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-mariadb.yaml updated with new image tags 2019-01-28 17:33:55.582 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-memcached.yaml has no img tags 2019-01-28 17:33:55.583 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-memcached.yaml updated with new image tags 2019-01-28 17:33:55.614 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-rabbitmq.yaml updated with new image tags 2019-01-28 17:33:55.742 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-keystone.yaml updated with new image tags 2019-01-28 17:33:55.839 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-glance.yaml updated with new image tags 2019-01-28 17:33:55.958 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-cinder.yaml updated with new image tags 2019-01-28 17:33:55.978 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-libvirt.yaml updated with new image tags 2019-01-28 17:33:56.210 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-nova.yaml updated with new image tags 2019-01-28 17:33:56.245 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-nova-api-proxy.yaml updated with new image tags 2019-01-28 17:33:56.533 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-neutron.yaml updated with new image tags 2019-01-28 17:33:56.648 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-heat.yaml updated with new image tags 2019-01-28 17:33:56.714 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-aodh.yaml updated with new image tags 2019-01-28 17:33:56.784 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-gnocchi.yaml updated with new image tags 2019-01-28 17:33:56.838 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-panko.yaml updated with new image tags 2019-01-28 17:33:57.053 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-ceilometer.yaml updated with new image tags 2019-01-28 17:33:57.220 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-horizon.yaml updated with new image tags 2019-01-28 17:33:57.238 11851 INFO sysinv.conductor.kube_app [-] Application (stx-openstack) upload completed. 2019-01-28 17:34:43.994 11851 INFO sysinv.conductor.kube_app [-] Application (stx-openstack) apply started. 2019-01-28 17:34:44.006 11851 ERROR sysinv.db.sqlalchemy.api [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] Failed to add host label openvswitch. Already exists with this uuid 2019-01-28 17:34:44.025 11851 ERROR sysinv.db.sqlalchemy.api [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] Failed to add host label openstack-control-plane. Already exists with this uuid 2019-01-28 17:34:44.028 11851 ERROR sysinv.db.sqlalchemy.api [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] Failed to add host label openstack-compute-node. Already exists with this uuid 2019-01-28 17:34:44.248 11851 INFO sysinv.conductor.kube_app [-] Generating application overrides... 2019-01-28 17:34:44.491 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_neutron_ml2_config={'ml2': {'type_drivers': 'managed_flat,managed_vlan,managed_vxlan', 'tenant_network_types': 'vlan,vxlan', 'mechanism_drivers': 'openvswitch,sriovnicswitch,l2population', 'path_mtu': 0, 'physical_network_mtus': 'physnet0:1500,physnet1:1500'}, 'securitygroup': {'firewall_driver': 'noop'}} 2019-01-28 17:34:44.554 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:34:44.646 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:34:44.767 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:34:44.783 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:34:44.908 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:34:44.924 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:34:45.045 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:34:45.061 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:34:45.179 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet0 2019-01-28 17:34:45.194 11851 INFO sysinv.helm.neutron [req-e04710ad-fecc-437e-9363-ec7b9e338f02 admin admin] _get_dynamic_ovs_agent_config datanet physnet1 2019-01-28 17:34:45.883 11851 INFO sysinv.conductor.kube_app [-] Application overrides generated. 2019-01-28 17:34:45.917 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/kube-system-ingress.yaml has no img tags 2019-01-28 17:34:45.919 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/kube-system-ingress.yaml updated with new image tags 2019-01-28 17:34:45.939 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-ingress.yaml has no img tags 2019-01-28 17:34:45.940 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-ingress.yaml updated with new image tags 2019-01-28 17:34:45.952 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-rbd-provisioner.yaml updated with new image tags 2019-01-28 17:34:45.982 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-mariadb.yaml updated with new image tags 2019-01-28 17:34:45.998 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-memcached.yaml has no img tags 2019-01-28 17:34:45.999 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-memcached.yaml updated with new image tags 2019-01-28 17:34:46.031 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-rabbitmq.yaml updated with new image tags 2019-01-28 17:34:46.154 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-keystone.yaml updated with new image tags 2019-01-28 17:34:46.250 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-glance.yaml updated with new image tags 2019-01-28 17:34:46.373 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-cinder.yaml updated with new image tags 2019-01-28 17:34:46.390 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-libvirt.yaml updated with new image tags 2019-01-28 17:34:46.609 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-nova.yaml updated with new image tags 2019-01-28 17:34:46.641 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-nova-api-proxy.yaml updated with new image tags 2019-01-28 17:34:46.830 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-neutron.yaml updated with new image tags 2019-01-28 17:34:47.031 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-heat.yaml updated with new image tags 2019-01-28 17:34:47.098 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-aodh.yaml updated with new image tags 2019-01-28 17:34:47.166 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-gnocchi.yaml updated with new image tags 2019-01-28 17:34:47.220 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-panko.yaml updated with new image tags 2019-01-28 17:34:47.428 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-ceilometer.yaml updated with new image tags 2019-01-28 17:34:47.592 11851 INFO sysinv.conductor.kube_app [-] Overrides file /opt/platform/helm/19.01/openstack-horizon.yaml updated with new image tags 2019-01-28 17:34:47.602 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-glance:dev-centos-pike-latest download started from local registry 2019-01-28 17:34:47.603 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/osixia/keepalived:1.4.5 download started from local registry 2019-01-28 17:34:47.604 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-aodh:dev-centos-pike-latest download started from local registry 2019-01-28 17:34:47.606 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/mariadb:10.2.13 download started from local registry 2019-01-28 17:34:47.607 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0 download started from local registry 2019-01-28 17:34:47.608 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-gnocchi:dev-centos-pike-latest download started from local registry 2019-01-28 17:34:47.609 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/prom/mysqld-exporter:v0.10.0 download started from local registry 2019-01-28 17:34:47.610 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-panko:dev-centos-pike-latest download started from local registry 2019-01-28 17:34:47.611 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-libvirt:dev-centos-pike-latest download started from local registry 2019-01-28 17:34:47.612 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/rabbitmq:3.7.4-management download started from local registry 2019-01-28 17:34:47.613 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/prom/memcached-exporter:v0.4.1 download started from local registry 2019-01-28 17:34:47.614 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-heat:dev-centos-pike-latest download started from local registry 2019-01-28 17:34:47.615 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/port/ceph-config-helper:v1.10.3 download started from local registry 2019-01-28 17:34:47.616 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/openstackhelm/mariadb:10.2.18 download started from local registry 2019-01-28 17:34:47.618 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-cinder:dev-centos-pike-latest download started from local registry 2019-01-28 17:34:47.619 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/rabbitmq:3.7.4 download started from local registry 2019-01-28 17:34:47.620 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-keystone:dev-centos-pike-latest download started from local registry 2019-01-28 17:34:47.623 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/rabbitmq:3.7-management download started from local registry 2019-01-28 17:34:47.624 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/quay.io/stackanetes/kubernetes-entrypoint:v0.3.1 download started from local registry 2019-01-28 17:34:47.625 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/kbudde/rabbitmq-exporter:v0.21.0 download started from local registry 2019-01-28 17:35:02.837 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-gnocchi:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.838 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-libvirt:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.838 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-ceph-config-helper:dev-centos-pike-latest download started from local registry 2019-01-28 17:35:02.840 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/docker:17.07.0 download started from local registry 2019-01-28 17:35:02.842 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/port/ceph-config-helper:v1.10.3 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.842 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-glance:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.843 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/rabbitmq:3.7.4 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.843 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-ceilometer:dev-centos-pike-latest download started from local registry 2019-01-28 17:35:02.849 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/kbudde/rabbitmq-exporter:v0.21.0 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.849 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-aodh:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.850 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/rabbitmq:3.7.4-management download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.854 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/openstackhelm/mariadb:10.2.18 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.861 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/rabbitmq:3.7-management download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.864 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/quay.io/stackanetes/kubernetes-entrypoint:v0.3.1 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.901 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/osixia/keepalived:1.4.5 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.901 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/gcr.io/google_containers/defaultbackend:1.0 download started from local registry 2019-01-28 17:35:02.902 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/memcached:1.5.5 download started from local registry 2019-01-28 17:35:02.908 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/mariadb:10.2.13 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.909 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-panko:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.909 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/kolla/ubuntu-source-nova-novncproxy:ocata download started from local registry 2019-01-28 17:35:02.911 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-heat:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.916 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-keystone:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.926 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/prom/memcached-exporter:v0.4.1 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.926 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/prom/mysqld-exporter:v0.10.0 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.927 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:02.927 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/kolla/ubuntu-source-rally:4.0.0 download started from local registry 2019-01-28 17:35:02.928 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-horizon:dev-centos-pike-latest download started from local registry 2019-01-28 17:35:02.929 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-nova:dev-centos-pike-latest download started from local registry 2019-01-28 17:35:02.930 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/quay.io/external_storage/rbd-provisioner:latest download started from local registry 2019-01-28 17:35:02.932 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-neutron:dev-centos-pike-latest download started from local registry 2019-01-28 17:35:02.933 11851 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-nova-api-proxy:dev-centos-pike-latest download started from local registry 2019-01-28 17:35:02.934 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-cinder:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.181 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-nova-api-proxy:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.182 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-nova:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.182 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-ceph-config-helper:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.196 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-ceilometer:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.197 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-neutron:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.198 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/kolla/ubuntu-source-rally:4.0.0 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.255 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/docker:17.07.0 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.256 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/memcached:1.5.5 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.257 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/starlingx/stx-horizon:dev-centos-pike-latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.270 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/quay.io/external_storage/rbd-provisioner:latest download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.271 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/kolla/ubuntu-source-nova-novncproxy:ocata download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.272 11851 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/gcr.io/google_containers/defaultbackend:1.0 download failed from local registry: 500 Server Error: Internal Server Error ("Get http://192.168.204.2:9001/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)") 2019-01-28 17:35:18.272 11851 ERROR sysinv.conductor.kube_app [-] Deployment of application stx-openstack failed: failed to download one or more image(s). 2019-01-28 17:35:18.272 11851 TRACE sysinv.conductor.kube_app Traceback (most recent call last): 2019-01-28 17:35:18.272 11851 TRACE sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 898, in perform_app_apply 2019-01-28 17:35:18.272 11851 TRACE sysinv.conductor.kube_app self._download_images(app) 2019-01-28 17:35:18.272 11851 TRACE sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 471, in _download_images 2019-01-28 17:35:18.272 11851 TRACE sysinv.conductor.kube_app reason="failed to download one or more image(s).") 2019-01-28 17:35:18.272 11851 TRACE sysinv.conductor.kube_app KubeAppApplyFailure: Deployment of application stx-openstack failed: failed to download one or more image(s). 2019-01-28 17:35:18.272 11851 TRACE sysinv.conductor.kube_app 2019-01-28 17:35:18.276 11851 ERROR sysinv.conductor.kube_app [-] Application apply aborted!. 2019-01-28 17:36:46.834 9134 INFO sysinv.agent.lldp.drivers.ovs.driver [-] Adding missing LLDP flow from lldp0b435963-e0 to eth1 2019-01-28 17:36:46.848 9134 INFO sysinv.agent.lldp.drivers.ovs.driver [-] Adding missing LLDP flow from lldp2e944997-9b to eth0 2019-01-28 17:41:21.247 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-41-21 patch 2019-01-28 17:41:21.248 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_end. No changes from mtce/1.0. 2019-01-28 17:46:21.278 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-46-21 patch 2019-01-28 17:46:21.279 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_end. No changes from mtce/1.0. 2019-01-28 17:51:21.316 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_start_2019-01-28-17-51-21 patch 2019-01-28 17:51:21.317 12613 INFO sysinv.api.controllers.v1.host [-] controller-0 ihost_patch_end. No changes from mtce/1.0.