error when creating kube-coredns

Bug #1798323 reported by Robert Duncan
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
magnum (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

Magnum as deployed by kolla-ansible 6.1.0 in Openstack queens.

magnum-api --version
6.1.1

openstack coe cluster template show k8s
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| insecure_registry | - |
| labels | {} |
| updated_at | 2018-10-16T16:32:47+00:00 |
| floating_ip_enabled | True |
| fixed_subnet | 1a2721a3-6974-4391-a742-740fb119f34f |
| master_flavor_id | m1.large |
| uuid | 103f51c0-e5c5-4a1b-ba57-1bea3637b7de |
| no_proxy | - |
| https_proxy | - |
| tls_disabled | True |
| keypair_id | mykey |
| public | False |
| http_proxy | - |
| docker_volume_size | 15 |
| server_type | vm |
| external_network_id | a3eacd21-2d88-40f1-9b67-48adf848a11c |
| cluster_distro | fedora-atomic |
| image_id | fedora-atomic-latest |
| volume_driver | cinder |
| registry_enabled | False |
| docker_storage_driver | devicemapper |
| apiserver_port | - |
| name | k8s |
| created_at | 2018-10-08T16:55:26+00:00 |
| network_driver | flannel |
| fixed_network | a601e6e0-4c7c-4a6e-a911-de8ed8258b15 |
| coe | kubernetes |
| flavor_id | m1.large |
| master_lb_enabled | False |
| dns_nameserver | 8.8.8.8 |
+-----------------------+--------------------------------------+

The fedora atomic image is the one recommended in the user guide, such as:

 openstack image create --container-format bare --disk-format raw --public --file=Fedora-Atomic-27-20180212.2.x86_64.qcow2 --property os_distro='fedora-atomic' fedora-atomic-latest

I have tried several variations of the Fedora atomic image with varying results, however I cannot get a kubernets master to bootstrap, here's the console log: heat does not spawn any minions and the stack eventually times out after 60 minutes with create failed.

There are a few errors, specifically:

this one is concerning me most:
##############
"
kubectl apply --validate=false -f -
[ 193.047828] cloud-init[1250]: error: unable to decode "STDIN": no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1beta1"
"

&&
I think this one is erroneous
###############

" 60.259770] cloud-init[1250]: Pulling layer 3bb67df590fb7166176759e564de2766df68cf5458a1b145c8a28330614fb5e9
[ 90.400104] cloud-init[1250]: g-io-error-quark: fallocate: No space left on device (12)
[ 90.403178] cloud-init[1250]: Pulling layer 846750b50aa3d195d902fb91579b24cd29fabf05d15eb012c8436714097a4cbc
"
and finally

###############

[ 193.839453] cloud-init[1250]: 2018-10-16 17:06:36,208 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
[ 193.840781] cloud-init[1250]: 2018-10-16 17:06:36,209 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3.5/site-packages/cloudinit/config/cc_scripts_user.py'>) failed

here's the complete log:

[ 1.489018] dracut-pre-udev[333]: rpcbind: /run/rpcbind/rpcbind.lock: No such file or directory
[ 1.502082] dracut-pre-udev[333]: rpc.idmapd: conf_reinit: open ("(null)", O_RDONLY) failed
[ 1.505126] dracut-pre-udev[333]: rpc.idmapd: conf_reinit: open ("(null)", O_RDONLY) failed
[ 1.507960] dracut-pre-udev[333]: rpc.idmapd: Could not find user "nobody"
%G%G%G%G[ 7.666169] cloud-init[758]: Cloud-init v. 0.7.9 running 'init-local' at Tue, 16 Oct 2018 17:03:29 +0000. Up 7.04 seconds.
[[0;32m OK [0m] Started Initial cloud-init job (pre-networking).
[[0;32m OK [0m] Reached target Network (Pre).
[[0;32m OK [0m] Reached target System Initialization.
[[0;32m OK [0m] Listening on Open-iSCSI iscsiuio Socket.
[[0;32m OK [0m] Listening on D-Bus System Message Bus Socket.
[[0;32m OK [0m] Listening on Open-iSCSI iscsid Socket.
[[0;32m OK [0m] Reached target Sockets.
[[0;32m OK [0m] Reached target Basic System.
         Starting NTP client/server...
         Starting Login Service...
         Starting OpenSSH rsa Server Key Generation...
[[0;32m OK [0m] Started Daily Cleanup of Temporary Directories.
[[0;32m OK [0m] Reached target Timers.
         Starting GSSAPI Proxy Daemon...
         Starting OpenSSH ed25519 Server Key Generation...
         Starting OpenSSH ecdsa Server Key Generation...
[[0;32m OK [0m] Started D-Bus System Message Bus.
         Starting Network Manager...
[[0;32m OK [0m] Started GSSAPI Proxy Daemon.
[[0;32m OK [0m] Started OpenSSH ed25519 Server Key Generation.
[[0;32m OK [0m] Started OpenSSH ecdsa Server Key Generation.
[[0;32m OK [0m] Reached target NFS client services.
[[0;32m OK [0m] Started Login Service.
[[0;32m OK [0m] Started NTP client/server.
[[0;32m OK [0m] Started OpenSSH rsa Server Key Generation.
[[0;32m OK [0m] Reached target sshd-keygen.target.
[[0;32m OK [0m] Started Network Manager.
[[0;32m OK [0m] Reached target Network.
[[0;32m OK [0m] Started Containerd Standalone OCI Container Daemon.
[[0;32m OK [0m] Started Logout off all iSCSI sessions on shutdown.
[[0;32m OK [0m] Reached target Remote File Systems (Pre).
[[0;32m OK [0m] Reached target Remote File Systems.
         Starting Initial cloud-init job (metadata service crawler)...
         Starting Network Manager Wait Online...
         Starting Network Manager Script Dispatcher Service...
[[0;32m OK [0m] Started Network Manager Script Dispatcher Service.
         Starting Hostname Service...
[[0;32m OK [0m] Started Hostname Service.
         Starting Authorization Manager...
[[0;32m OK [0m] Started Authorization Manager.
[[0;32m OK [0m] Started Network Manager Wait Online.
[ 13.818047] cloud-init[903]: Cloud-init v. 0.7.9 running 'init' at Tue, 16 Oct 2018 17:03:31 +0000. Up 9.07 seconds.
[ 13.819344] cloud-init[903]: ci-info: +++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++
[ 13.819874] cloud-init[903]: ci-info: +--------+------+-----------+---------------+-------+-------------------+
[ 13.821020] cloud-init[903]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
[ 13.822092] cloud-init[903]: ci-info: +--------+------+-----------+---------------+-------+-------------------+
[ 13.823176] cloud-init[903]: ci-info: | eth0: | True | 10.0.0.16 | 255.255.255.0 | . | fa:16:3e:25:ea:7c |
[ 13.824298] cloud-init[903]: ci-info: | eth0: | True | . | . | d | fa:16:3e:25:ea:7c |
[ 13.830120] cloud-init[903]: ci-info: | lo: | True | 127.0.0.1 | 255.0.0.0 | . | . |
[ 13.830651] cloud-init[903]: ci-info: | lo: | True | . | . | d | . |
[ 13.831032] cloud-init[903]: ci-info: +--------+------+-----------+---------------+-------+-------------------+
[ 13.831329] cloud-init[903]: ci-info: ++++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++++
[ 13.831669] cloud-init[903]: ci-info: +-------+-----------------+----------+-----------------+-----------+-------+
[ 13.832060] cloud-init[903]: ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
[ 13.832365] cloud-init[903]: ci-info: +-------+-----------------+----------+-----------------+-----------+-------+
[ 13.832722] cloud-init[903]: ci-info: | 0 | 0.0.0.0 | 10.0.0.1 | 0.0.0.0 | eth0 | UG |
[ 13.833080] cloud-init[903]: ci-info: | 1 | 10.0.0.0 | 0.0.0.0 | 255.255.255.0 | eth0 | U |
[ 13.833382] cloud-init[903]: ci-info: | 2 | 169.254.169.254 | 10.0.0.2 | 255.255.255.255 | eth0 | UGH |
[ 13.833737] cloud-init[903]: ci-info: +-------+-----------------+----------+-----------------+-----------+-------+
[[0;32m OK [0m] Started Initial cloud-init job (metadata service crawler).
[[0;32m OK [0m] Reached target Network is Online.
         Starting Notify NFS peers of a restart...
         Starting Permit User Sessions...
         Starting Docker Storage Setup...
[[0;32m OK [0m] Reached target Cloud-config availability.
         Starting Apply the settings specified in cloud-config...
         Starting OpenSSH server daemon...
[[0;32m OK [0m] Started Notify NFS peers of a restart.
[[0;32m OK [0m] Started Permit User Sessions.
         Starting Hold until boot process finishes up...
         Starting Terminate Plymouth Boot Screen...
[ 15.124085] cloud-init[1079]: Cloud-init v. 0.7.9 running 'modules:config' at Tue, 16 Oct 2018 17:03:37 +0000. Up 14.91 seconds.
[ 16.521199] cloud-init[1250]: + CA_FILE=/etc/pki/ca-trust/source/anchors/openstack-ca.pem
[ 16.525360] cloud-init[1250]: + '[' -n '' ']'

Fedora 25 (Atomic Host)
Kernel 4.11.10-200.fc25.x86_64 on an x86_64 (ttyS0)

djyhftj-bpu2zxlywylj-master-0 login: [ 23.873903] cloud-init[1250]: Pulling layer 7ecc8a985be6848a7ee66b80217c168de6e79dcbda3c01ee36a45367c3a3a6bc
[ 23.875151] cloud-init[1250]: Pulling layer a6091bfb179ad957897301750adebbb3dc821e0aff7e2695e1e87128f91d705c
[ 23.876226] cloud-init[1250]: Pulling layer a6220ca8d39b93c71aa73bc81d83160f4c414f1ca6688b79a4e343a0f1ec5973
[ 23.876959] cloud-init[1250]: Pulling layer 76cd47524e799b1197a4a641ec9fe2c102265d5f188cacadf07bfda2bee5da0d
[ 23.877779] cloud-init[1250]: Pulling layer cde34beaf6309f264b84a33e37a4728cca16d21e2bb5fd1bf45140ea72cfde03
[ 23.878612] cloud-init[1250]: Pulling layer e3bab3136514f51d8e0082f241ba3aec260150a1cee7e6cdac36e6a2e2377951
[ 23.879054] cloud-init[1250]: Pulling layer 5daa2b6fae719323bda2d879fe3c101afc140265482f7712d2971858d88e291e
[ 50.951481] cloud-init[1250]: Pulling layer 3ba53a147ecd07b7212b1a1437b5a1c76fdf2b108df17e8396e735b2d251f41b
[ 52.372526] cloud-init[1250]: Extracting to /var/lib/containers/atomic/etcd.0
[ 52.378880] cloud-init[1250]: File /etc/etcd/etcd.conf already exists.
[ 52.493048] cloud-init[1250]: systemctl daemon-reload
[ 52.508441] cloud-init[1250]: systemd-tmpfiles --create /etc/tmpfiles.d/etcd.conf
[ 52.624581] cloud-init[1250]: systemctl enable etcd
[ 52.879419] cloud-init[1250]: Removed /etc/systemd/system/multi-user.target.wants/docker-storage-setup.service.
[ 53.126407] cloud-init[1250]: Physical volume "/dev/disk/by-id/virtio-05478377-bc61-445f-9" successfully created.
[ 53.272717] cloud-init[1250]: Volume group "docker" successfully created
[ 53.447634] cloud-init[1250]: Rounding up size to full physical extent 16.00 MiB
[ 53.550885] cloud-init[1250]: Logical volume "docker-poolmeta" created.
[ 53.720951] cloud-init[1250]: Logical volume "docker-pool" created.
[ 53.742728] cloud-init[1250]: WARNING: Converting logical volume docker/docker-pool and docker/docker-poolmeta to thin pool's data and metadata volumes with metadata wiping.
[ 53.743115] cloud-init[1250]: THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
[ 54.140078] cloud-init[1250]: Converted docker/docker-pool to thin pool.
[ 54.373702] cloud-init[1250]: Logical volume docker/docker-pool changed.
[ 54.385048] cloud-init[1250]: configuring kubernetes (master)
[ 60.255121] cloud-init[1250]: Pulling layer 3463131de5b085ae9a30c13d5c526f9297e0fe72df17fa141a1765cb371015fd
[ 60.256963] cloud-init[1250]: Pulling layer d5fb7cc999272ece23da54e91e99a3112762fb7795e484f6d0417b7f7de23e2b
[ 60.257725] cloud-init[1250]: Pulling layer 7e8cdf9900e2a4b61f2e25c9b5ba77fc84f3fba44ed9304810d5570a40c0b3db
[ 60.258146] cloud-init[1250]: Pulling layer 1d16a9f678250828a8d1b2d287bc0d0c18a5675596d63bd9cb6411a872e6ad9c
[ 60.258517] cloud-init[1250]: Pulling layer 34d2bce38fe0837669be1e3c7f77edcce5bcfb96d2f7e185a95e56d7e5bccf06
[ 60.258960] cloud-init[1250]: Pulling layer 87ef82017b811187513a6c37b5a9d1b657b5d4c79eea3dd9e4d1863a10532144
[ 60.259770] cloud-init[1250]: Pulling layer 3bb67df590fb7166176759e564de2766df68cf5458a1b145c8a28330614fb5e9
[ 90.400104] cloud-init[1250]: g-io-error-quark: fallocate: No space left on device (12)
[ 90.403178] cloud-init[1250]: Pulling layer 846750b50aa3d195d902fb91579b24cd29fabf05d15eb012c8436714097a4cbc
[ 96.339185] cloud-init[1250]: Pulling layer 3463131de5b085ae9a30c13d5c526f9297e0fe72df17fa141a1765cb371015fd
[ 96.339830] cloud-init[1250]: Pulling layer d5fb7cc999272ece23da54e91e99a3112762fb7795e484f6d0417b7f7de23e2b
[ 96.341226] cloud-init[1250]: Pulling layer 7e8cdf9900e2a4b61f2e25c9b5ba77fc84f3fba44ed9304810d5570a40c0b3db
[ 96.342840] cloud-init[1250]: Pulling layer e2159df0a65a19c5dae53ca4692d7be7e629d264f922ed6b2478a6d0c8541dbd
[ 96.345098] cloud-init[1250]: Pulling layer 108ac72bedc5a771b79695b7610f858f084527d0c0ec6111e6138c9ad43f106b
[ 110.585438] cloud-init[1250]: g-io-error-quark: min-free-space-percent '3%' would be exceeded, 149.3 MB more required (0)
[ 110.588356] cloud-init[1250]: Pulling layer 85be7c2510a4a87225e74b3927b5bd30c483dc3527bd21d7063a1a0c091037b4
[ 116.438087] cloud-init[1250]: Pulling layer 3463131de5b085ae9a30c13d5c526f9297e0fe72df17fa141a1765cb371015fd
[ 116.447435] cloud-init[1250]: Pulling layer d5fb7cc999272ece23da54e91e99a3112762fb7795e484f6d0417b7f7de23e2b
[ 116.448302] cloud-init[1250]: Pulling layer 7e8cdf9900e2a4b61f2e25c9b5ba77fc84f3fba44ed9304810d5570a40c0b3db
[ 116.449056] cloud-init[1250]: Pulling layer 4c14709b7f4a00b74a2cfc2b80ec5a3d4e75d8b26335548b364057aa9f30a8aa
[ 116.449482] cloud-init[1250]: Pulling layer ef8fcf929d9f6c0034f5414195b6adf5961b11c5b4096753fa28b5352e677027
[ 137.191555] cloud-init[1250]: g-io-error-quark: min-free-space-percent '3%' would be exceeded, 149.3 MB more required (0)
[ 137.195413] cloud-init[1250]: Pulling layer 8bdfb179a1d2b55cb6e4af1062c5e8cfd89d8116fd56f06138f4d0b0b5b74ead
[ 143.070160] cloud-init[1250]: Pulling layer 61b7c97279a555e18bf110b78ab39ba1dea16bad4b88756fd7f1900b789eccb9
[ 143.075037] cloud-init[1250]: Pulling layer c80ade50ed9043de1461bd622bf95339a59894c68191360dcc4ea28a5d267019
[ 143.075784] cloud-init[1250]: Pulling layer 3a7e1b8b1915be0197b045e8d36ca4e3e81299d941aaeedaceb5a769dd98096c
[ 143.076626] cloud-init[1250]: Pulling layer 8e60b5f5e8894c88f2af29a4fde95139d7aed18680a86e6c7945a03d0952359a
[ 152.495472] cloud-init[1250]: Pulling layer 55da6b08c07beab88b3baa58b502a6e324fd2945e08fb9722ce033e5a39a570d
[ 153.094621] cloud-init[1250]: Extracting to /var/lib/containers/atomic/flanneld.0
[ 153.099950] cloud-init[1250]: File /etc/sysconfig/flanneld already exists.
[ 153.211494] cloud-init[1250]: systemctl daemon-reload
[ 153.226880] cloud-init[1250]: systemd-tmpfiles --create /etc/tmpfiles.d/flanneld.conf
[ 153.314729] cloud-init[1250]: systemctl enable flanneld
[ 153.417449] cloud-init[1250]: + _prefix=docker.io/openstackmagnum/
[ 153.417951] cloud-init[1250]: + atomic install --storage ostree --system --system-package no --set REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt --name heat-container-agent docker.io/openstackmagnum/heat-container-agent:rawhide
[ 159.246123] cloud-init[1250]: Pulling layer 32d051af12f8fb4de96468f9d1dd7e2045444820f84aa53e318c5d23dc9481e7
[ 159.246556] cloud-init[1250]: Pulling layer 086aa3838e9f52e3a6aeb83998f8d87e5f4f1208ceb4655b9b5a7d21c696bc72
[ 159.246950] cloud-init[1250]: Pulling layer 35c6835c61aad2abb02ab746b0fd22860e205fa32922163a7f5269aa63410d32
[ 159.247818] cloud-init[1250]: Pulling layer 938954ceff8876e4187da19479fe63c49d19c3bb31ec401c20b2279f41d5eb05
[ 159.248682] cloud-init[1250]: Pulling layer c7a2341b649502b7b9b48173415f1db2f6fc6ea5baea7c4ed58538c9c5c29f5c
[ 159.249177] cloud-init[1250]: Pulling layer d119e0ba16d60a3dd6f7403f15d2508300997de6963f4449cc1b42d04b9ab4a8
[ 159.249580] cloud-init[1250]: Pulling layer a62589cc8c25666193ab76056c371e36a770b8bc385d89ae3b67473b9c12fe29
[ 159.249976] cloud-init[1250]: Pulling layer ad40c553bd1bcb0746a9d6b6a1df7559faa06b0c70a0ff7c34678e892ea11021
[ 159.250838] cloud-init[1250]: Pulling layer e4e5d7a594921460060d6ea9f344ae9cf719d09e0fd179e2ee2a40f666883bd2
[ 159.251683] cloud-init[1250]: Pulling layer 6261e53e3aca9d7068a2c4ec2c631b34be4d356d66c3d927051433cab9f1e5d1
[ 159.252122] cloud-init[1250]: Pulling layer 1934d0968e292f159906c152602ce0dfc42304786647d3332e88fe08a527cc1b
[ 159.252537] cloud-init[1250]: Pulling layer e5b6be0df0154e6d0f7749f295393331951ce47f6ef4ef5a3e1b081078e42932
[ 159.252972] cloud-init[1250]: Pulling layer 15eb7a13273fa00596131d4734c1610c9472685da81f7fb1a2ebcadf212bc84c
[ 184.881581] cloud-init[1250]: Pulling layer 96ce1eb42495f7bbf766c2b4049858fabf7907831888a2d90e916c5aa41054c4
[ 186.192413] cloud-init[1250]: Extracting to /var/lib/containers/atomic/heat-container-agent.0
[ 186.317027] cloud-init[1250]: systemctl daemon-reload
[ 186.330307] cloud-init[1250]: systemd-tmpfiles --create /etc/tmpfiles.d/heat-container-agent.conf
[ 186.430483] cloud-init[1250]: systemctl enable heat-container-agent
[ 186.497279] cloud-init[1250]: + systemctl start heat-container-agent
[ 186.639316] cloud-init[1250]: starting services
[ 186.649442] cloud-init[1250]: activating service etcd
[ 186.811547] cloud-init[1250]: activating service docker
[ 187.019733] cloud-init[1250]: activating service kube-apiserver
[ 187.032982] cloud-init[1250]: Created symlink /etc/systemd/system/multi-user.target.wants/kube-apiserver.service ��� /usr/lib/systemd/system/kube-apiserver.service.
[ 187.147088] cloud-init[1250]: activating service kube-controller-manager
[ 187.153995] cloud-init[1250]: Created symlink /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service ��� /usr/lib/systemd/system/kube-controller-manager.service.
[ 187.290709] cloud-init[1250]: activating service kube-scheduler
[ 187.309902] cloud-init[1250]: Created symlink /etc/systemd/system/multi-user.target.wants/kube-scheduler.service ��� /usr/lib/systemd/system/kube-scheduler.service.
[ 187.477585] cloud-init[1250]: creating /usr/local/bin/flannel-config
[ 187.500658] cloud-init[1250]: Created symlink /etc/systemd/system/multi-user.target.wants/flannel-config.service ��� /etc/systemd/system/flannel-config.service.
[ 187.648086] cloud-init[1250]: activating service flanneld
[ 187.757753] cloud-init[1250]: + . /etc/sysconfig/heat-params
[ 187.760763] cloud-init[1250]: ++ PROMETHEUS_MONITORING=False
[ 187.761075] cloud-init[1250]: ++ KUBE_API_PUBLIC_ADDRESS=87.44.4.55
[ 187.761413] cloud-init[1250]: ++ KUBE_API_PRIVATE_ADDRESS=10.0.0.16
[ 187.761789] cloud-init[1250]: ++ KUBE_API_PORT=8080
[ 187.762619] cloud-init[1250]: ++ KUBE_NODE_PUBLIC_IP=87.44.4.55
[ 187.763016] cloud-init[1250]: ++ KUBE_NODE_IP=10.0.0.16
[ 187.763352] cloud-init[1250]: ++ KUBE_ALLOW_PRIV=true
[ 187.763788] cloud-init[1250]: ++ ENABLE_CINDER=
[ 187.764846] cloud-init[1250]: ++ ETCD_VOLUME=c4572e10-30ae-4f4f-9e1d-08488e4e3f22
[ 187.765649] cloud-init[1250]: ++ ETCD_VOLUME_SIZE=0
[ 187.766078] cloud-init[1250]: ++ DOCKER_VOLUME=05478377-bc61-445f-95a7-117440d58380
[ 187.766421] cloud-init[1250]: ++ DOCKER_VOLUME_SIZE=15
[ 187.766816] cloud-init[1250]: ++ DOCKER_STORAGE_DRIVER=devicemapper
[ 187.767596] cloud-init[1250]: ++ NETWORK_DRIVER=flannel
[ 187.768061] cloud-init[1250]: ++ FLANNEL_NETWORK_CIDR=10.100.0.0/16
[ 187.768412] cloud-init[1250]: ++ FLANNEL_NETWORK_SUBNETLEN=24
[ 187.768794] cloud-init[1250]: ++ FLANNEL_BACKEND=udp
[ 187.769162] cloud-init[1250]: ++ PODS_NETWORK_CIDR=10.100.0.0/16
[ 187.769621] cloud-init[1250]: ++ PORTAL_NETWORK_CIDR=10.254.0.0/16
[ 187.770078] cloud-init[1250]: ++ ADMISSION_CONTROL_LIST=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota
[ 187.770406] cloud-init[1250]: ++ ETCD_DISCOVERY_URL=https://discovery.etcd.io/1d5e6a47d28fe3a3e849ac85e8296692
[ 187.770735] cloud-init[1250]: ++ USERNAME=admin
[ 187.771111] cloud-init[1250]: ++ PASSWORD=ChangeMe
[ 187.771451] cloud-init[1250]: ++ CLUSTER_SUBNET=1a2721a3-6974-4391-a742-740fb119f34f
[ 187.771849] cloud-init[1250]: ++ TLS_DISABLED=True
[ 187.772551] cloud-init[1250]: ++ KUBE_DASHBOARD_ENABLED=True
[ 187.772876] cloud-init[1250]: ++ INFLUX_GRAFANA_DASHBOARD_ENABLED=False
[ 187.773622] cloud-init[1250]: ++ VERIFY_CA=True
[ 187.773968] cloud-init[1250]: ++ CLUSTER_UUID=14e93d3c-f172-4bd3-914d-c3606f7c7611
[ 187.774677] cloud-init[1250]: ++ MAGNUM_URL=https://################:9511/v1
[ 187.775485] cloud-init[1250]: ++ VOLUME_DRIVER=cinder
[ 187.775879] cloud-init[1250]: ++ HTTP_PROXY=
[ 187.776551] cloud-init[1250]: ++ HTTPS_PROXY=
[ 187.776903] cloud-init[1250]: ++ NO_PROXY=
[ 187.777613] cloud-init[1250]: ++ WAIT_CURL='curl -i -X POST -H '\''X-Auth-Token: gAAAAABbxhmQjMeEMI2URJotaK8Sqsirx5rYpZF4gxpmnvNOK0ah6Nx8s3JFwg8MZ-xFnAYbAYbK6bF-NElrkTpviVbne0ygg-EwgYMypiVs9Q9RpWwYR90DMzNGskd_nG89mX2Ry9rfkqP-TNxZHFsa32-oL-ctA7H-XlbkUNiarGCUdoQmYkI'\'' -H '\''Content-Type: application/json'\'' -H '\''Accept: application/json'\'' https://#################:8004/v1/f858dee6da724dcba43a17341af78c83/stacks/djyhftj-bpu2zxlywylj-kube_masters-wy47q3n75qvd-0-ysr6kncv7yym/f9f36fc3-9b22-4deb-92de-66c896e20e74/resources/master_wait_handle/signal'
[ 187.778033] cloud-init[1250]: ++ KUBE_TAG=v1.9.3
[ 187.778331] cloud-init[1250]: ++ ETCD_TAG=v3.2.7
[ 187.778660] cloud-init[1250]: ++ FLANNEL_TAG=v0.9.0
[ 187.779034] cloud-init[1250]: ++ KUBE_VERSION=v1.9.3
[ 187.779368] cloud-init[1250]: ++ KUBE_DASHBOARD_VERSION=v1.8.3
[ 187.779699] cloud-init[1250]: ++ TRUSTEE_USER_ID=######################
[ 187.780142] cloud-init[1250]: ++ TRUSTEE_PASSWORD=#################
[ 187.780397] cloud-init[1250]: ++ TRUST_ID=
[ 187.780773] cloud-init[1250]: ++ AUTH_URL=https://###########:5000/v3/
[ 187.781479] cloud-init[1250]: ++ INSECURE_REGISTRY_URL=
[ 187.781824] cloud-init[1250]: ++ CONTAINER_INFRA_PREFIX=
[ 187.782503] cloud-init[1250]: ++ SYSTEM_PODS_INITIAL_DELAY=30
[ 187.782796] cloud-init[1250]: ++ SYSTEM_PODS_TIMEOUT=5
[ 187.783525] cloud-init[1250]: ++ ETCD_LB_VIP=
[ 187.783824] cloud-init[1250]: ++ DNS_SERVICE_IP=10.254.0.10
[ 187.784481] cloud-init[1250]: ++ DNS_CLUSTER_DOMAIN=cluster.local
[ 187.784792] cloud-init[1250]: ++ CERT_MANAGER_API=False
[ 187.785469] cloud-init[1250]: ++ CA_KEY=
[ 187.785772] cloud-init[1250]: ++ CALICO_TAG=v2.6.7
[ 187.786468] cloud-init[1250]: ++ CALICO_CNI_TAG=v1.11.2
[ 187.786773] cloud-init[1250]: ++ CALICO_KUBE_CONTROLLERS_TAG=v1.0.3
[ 187.787439] cloud-init[1250]: ++ CALICO_IPV4POOL=192.168.0.0/16
[ 187.787741] cloud-init[1250]: ++ INGRESS_CONTROLLER=
[ 187.788117] cloud-init[1250]: ++ INGRESS_CONTROLLER_ROLE=ingress
[ 187.788443] cloud-init[1250]: ++ KUBELET_OPTIONS=
[ 187.788781] cloud-init[1250]: ++ KUBECONTROLLER_OPTIONS=
[ 187.789441] cloud-init[1250]: ++ KUBEAPI_OPTIONS=
[ 187.789831] cloud-init[1250]: ++ KUBEPROXY_OPTIONS=
[ 187.790488] cloud-init[1250]: ++ KUBESCHEDULER_OPTIONS=
[ 187.790800] cloud-init[1250]: + echo 'Waiting for Kubernetes API...'
[ 187.791487] cloud-init[1250]: Waiting for Kubernetes API...
[ 187.791788] cloud-init[1250]: + curl --silent http://127.0.0.1:8080/version
[ 187.792543] cloud-init[1250]: + sleep 5
[ 192.782106] cloud-init[1250]: + curl --silent http://127.0.0.1:8080/version
[ 192.799712] cloud-init[1250]: {
[ 192.800088] cloud-init[1250]: "major": "1",
[ 192.800708] cloud-init[1250]: "minor": "5",
[ 192.801792] cloud-init[1250]: "gitVersion": "v1.5.3",
[ 192.802164] cloud-init[1250]: "gitCommit": "029c3a408176b55c30846f0faedf56aae5992e9b",
[ 192.802847] cloud-init[1250]: "gitTreeState": "clean",
[ 192.803596] cloud-init[1250]: "buildDate": "2017-03-09T11:55:06Z",
[ 192.804645] cloud-init[1250]: "goVersion": "go1.7.5",
[ 192.805713] cloud-init[1250]: "compiler": "gc",
[ 192.806778] cloud-init[1250]: "platform": "linux/amd64"
[ 192.809382] cloud-init[1250]: }+ cat
[ 192.810143] cloud-init[1250]: + kubectl apply --validate=false -f -
[ 193.047828] cloud-init[1250]: error: unable to decode "STDIN": no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1beta1"
[ 193.050635] cloud-init[1250]: + cat
[ 193.051661] cloud-init[1250]: + kubectl apply --validate=false -f -
[ 193.242126] cloud-init[1250]: error: unable to decode "STDIN": no kind "ClusterRoleBinding" is registered for version "rbac.authorization.k8s.io/v1beta1"
[ 193.250669] cloud-init[1250]: + ADMIN_RBAC=/srv/magnum/kubernetes/kubernetes-admin-rbac.yaml
[ 193.253626] cloud-init[1250]: + '[' -f /srv/magnum/kubernetes/kubernetes-admin-rbac.yaml ']'
[ 193.256515] cloud-init[1250]: + echo 'Writing File: /srv/magnum/kubernetes/kubernetes-admin-rbac.yaml'
[ 193.260727] cloud-init[1250]: Writing File: /srv/magnum/kubernetes/kubernetes-admin-rbac.yaml
[ 193.264764] cloud-init[1250]: ++ dirname /srv/magnum/kubernetes/kubernetes-admin-rbac.yaml
[ 193.268792] cloud-init[1250]: + mkdir -p /srv/magnum/kubernetes
[ 193.271951] cloud-init[1250]: + cat
[ 193.284791] cloud-init[1250]: Writing File: /etc/kubernetes/manifests/kube-coredns.yaml
[ 193.293474] cloud-init[1250]: Waiting for Kubernetes API...
[ 193.306098] cloud-init[1250]: {
[ 193.308092] cloud-init[1250]: "major": "1",
[ 193.310324] cloud-init[1250]: "minor": "5",
[ 193.312338] cloud-init[1250]: "gitVersion": "v1.5.3",
[ 193.314661] cloud-init[1250]: "gitCommit": "029c3a408176b55c30846f0faedf56aae5992e9b",
[ 193.317711] cloud-init[1250]: "gitTreeState": "clean",
[ 193.320095] cloud-init[1250]: "buildDate": "2017-03-09T11:55:06Z",
[ 193.322645] cloud-init[1250]: "goVersion": "go1.7.5",
[ 193.324783] cloud-init[1250]: "compiler": "gc",
[ 193.326843] cloud-init[1250]: "platform": "linux/amd64"
[ 193.513813] cloud-init[1250]: }serviceaccount "coredns" created
[ 193.530531] cloud-init[1250]: configmap "coredns" created
[ 193.551350] cloud-init[1250]: deployment "coredns" created
[ 193.597106] cloud-init[1250]: service "kube-dns" created
[ 193.630364] cloud-init[1250]: serviceaccount "kube-dns-autoscaler" created
[ 193.645795] cloud-init[1250]: Error from server (BadRequest): error when creating "/etc/kubernetes/manifests/kube-coredns.yaml": ClusterRole in version "v1beta1" cannot be handled as a ClusterRole: no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1beta1"
[ 193.654226] cloud-init[1250]: Error from server (BadRequest): error when creating "/etc/kubernetes/manifests/kube-coredns.yaml": ClusterRoleBinding in version "v1beta1" cannot be handled as a ClusterRoleBinding: no kind "ClusterRoleBinding" is registered for version "rbac.authorization.k8s.io/v1beta1"
[ 193.661238] cloud-init[1250]: Error from server (BadRequest): error when creating "/etc/kubernetes/manifests/kube-coredns.yaml": ClusterRole in version "v1" cannot be handled as a ClusterRole: no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1"
[ 193.667421] cloud-init[1250]: Error from server (BadRequest): error when creating "/etc/kubernetes/manifests/kube-coredns.yaml": ClusterRoleBinding in version "v1" cannot be handled as a ClusterRoleBinding: no kind "ClusterRoleBinding" is registered for version "rbac.authorization.k8s.io/v1"
[ 193.676973] cloud-init[1250]: unable to recognize "/etc/kubernetes/manifests/kube-coredns.yaml": no matches for apps/, Kind=Deployment
[ 193.690191] cloud-init[1250]: Cloud-init v. 0.7.9 running 'modules:final' at Tue, 16 Oct 2018 17:03:38 +0000. Up 16.31 seconds.
[ 193.695152] cloud-init[1250]: 2018-10-16 17:06:36,056 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-018 [1]
[ 193.730046] cloud-init[1250]: Created symlink /etc/systemd/system/multi-user.target.wants/wc-notify.service ��� /etc/systemd/system/wc-notify.service.
[ 193.839453] cloud-init[1250]: 2018-10-16 17:06:36,208 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
[ 193.840781] cloud-init[1250]: 2018-10-16 17:06:36,209 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3.5/site-packages/cloudinit/config/cc_scripts_user.py'>) failed
ci-info: ++++++Authorized keys from /home/fedora/.ssh/authorized_keys for user fedora+++++++
ci-info: +---------+-------------------------------------------------+---------+-----------+
ci-info: | Keytype | Fingerprint (md5) | Options | Comment |
ci-info: +---------+-------------------------------------------------+---------+-----------+
ci-info: | ssh-rsa | 97:97:5a:2d:64:cc:fb:7e:bf:e3:38:15:ba:2f:d3:95 | - | rob@kolla |
ci-info: +---------+-------------------------------------------------+---------+-----------+
<14>Oct 16 17:06:36 ec2:
<14>Oct 16 17:06:36 ec2: #############################################################
<14>Oct 16 17:06:36 ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
<14>Oct 16 17:06:36 ec2: 256 SHA256:5qCh5vKl8GDvwxSDfAAWlu1G4P7MNxMRYfplGr0U5Dg no comment (ECDSA)
<14>Oct 16 17:06:36 ec2: 256 SHA256:rB3dy+HnoQinecnh1FNp7qtccuFwsOKY1DGZT00tn9M no comment (ED25519)
<14>Oct 16 17:06:36 ec2: 2048 SHA256:cCWy6DZJeJqY1DaExG3MdATIu2hV/Y1vizdK+PTjMIo no comment (RSA)
<14>Oct 16 17:06:36 ec2: -----END SSH HOST KEY FINGERPRINTS-----
<14>Oct 16 17:06:36 ec2: #############################################################
-----BEGIN SSH HOST KEY KEYS-----
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPHKzVDV1w9Q5P1vxlMIiZvFTQzA+JUuQqlTBrtFFNwoOWbs5f/7RKxoWvpdqSTUGGz8fJX79Jsjn1ayVYD7Rs0=
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINfw+gC2EuVoWckcpmJU7dA12s6rmuSyx1MJpk4vMQds
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpdgTU1zJ85YKbbQVTC7oy5DrE6xQsu1Sv8JlHSEAYlCHpdtTBaSN78suxIF2SN2wPbN/NEGO59i+2UOXLx32DpxkDznFltARjgDMv++k8T6vuV1aBnJcnLJAVC0YaIWROGNAIjIYQV085RmefNvQmv6X0eRH+gvZNVOaGxpxkG1NTWGg1cugMxF30d3osZE8W051DSVTKDmdlzmB/a5KsXBA1MiAt6+DFKkGVD1GHB91BDJYq2eGYcmeXMhOEA/LMY95OOyOC4AUP9IfHfWil8aLBdXU7bjZrbEWRTW8LriGGmH7g/yp0RjLI99/MBswiYZFMACPDYPhZxa/LLx05
-----END SSH HOST KEY KEYS-----
[ 193.918840] cloud-init[1250]: Cloud-init v. 0.7.9 finished at Tue, 16 Oct 2018 17:06:36 +0000. Datasource DataSourceOpenStack [net,ver=2]. Up 193.90 seconds

after I ssh onto the master node:

####################
http://paste.openstack.org/show/732327/

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in magnum (Ubuntu):
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.