error getting node not found

Bug #1616156 reported by Daneyon Hansen
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Magnum
In Progress
Medium
Unassigned

Bug Description

I am using Magnum stable/mitaka branch. I can instantiate a k8s bay, but I see the following errors in the master kubelet log:

--- k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal ping statistics ---
14 packets transmitted, 14 received, 0% packet loss, time 13001ms
rtt min/avg/max/mdev = 0.025/0.033/0.050/0.008 ms

[minion@k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5 ~]$ sudo systemctl status kubelet -l
● kubelet.service - Kubernetes Kubelet Server
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled)
   Active: active (running) since Tue 2016-08-23 17:27:29 UTC; 4min 42s ago
     Docs: https://github.com/GoogleCloudPlatform/kubernetes
 Main PID: 1318 (kubelet)
   CGroup: /system.slice/kubelet.service
           ├─1318 /usr/bin/kubelet --logtostderr=true --v=0 --api_servers=http://127.0.0.1:8080 --address=0.0.0.0 --allow_privileged=true --register-node=false --config=/etc/kubernetes/manifests
           └─1341 journalctl -f

Aug 23 17:31:59 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:31:59.160906 1318 kubelet.go:1933] Error updating node status, will retry: error getting node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal": node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal" not found
Aug 23 17:31:59 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:31:59.163727 1318 kubelet.go:1933] Error updating node status, will retry: error getting node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal": node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal" not found
Aug 23 17:31:59 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:31:59.164302 1318 kubelet.go:839] Unable to update node status: update node status exceeds retry count
Aug 23 17:32:06 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:32:06.831733 1318 kubelet.go:1641] error getting node: node k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal not found
Aug 23 17:32:09 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:32:09.171889 1318 kubelet.go:1933] Error updating node status, will retry: error getting node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal": node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal" not found
Aug 23 17:32:09 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:32:09.178053 1318 kubelet.go:1933] Error updating node status, will retry: error getting node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal": node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal" not found
Aug 23 17:32:09 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:32:09.183941 1318 kubelet.go:1933] Error updating node status, will retry: error getting node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal": node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal" not found
Aug 23 17:32:09 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:32:09.188885 1318 kubelet.go:1933] Error updating node status, will retry: error getting node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal": node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal" not found
Aug 23 17:32:09 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:32:09.193694 1318 kubelet.go:1933] Error updating node status, will retry: error getting node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal": node "k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal" not found
Aug 23 17:32:09 k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal kubelet[1318]: E0823 17:32:09.194279 1318 kubelet.go:839] Unable to update node status: update node status exceeds retry count
[minion@k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5 ~]$ date
Tue Aug 23 17:32:13 UTC 2016

However, I can ping the node name:
[minion@k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5 ~]$ sudo cp /usr/bin/ping ~/ping
[minion@k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5 ~]$ sudo ~/ping k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal
PING k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6) 56(84) bytes of data.
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=1 ttl=64 time=0.025 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=2 ttl=64 time=0.029 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=3 ttl=64 time=0.032 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=4 ttl=64 time=0.032 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=5 ttl=64 time=0.031 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=6 ttl=64 time=0.050 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=7 ttl=64 time=0.033 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=8 ttl=64 time=0.033 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=9 ttl=64 time=0.034 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=10 ttl=64 time=0.037 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=11 ttl=64 time=0.027 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=12 ttl=64 time=0.033 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=13 ttl=64 time=0.030 ms
64 bytes from k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5.novalocal (10.0.0.6): icmp_seq=14 ttl=64 time=0.044 ms
^C

I am using the f-atomic-5 image:
[minion@k8-csc3nktiyz-0-i5bhydwhwz7q-kube-master-jd3fhyh23en5 ~]$ cat /etc/os-release
NAME=Fedora
VERSION="21 (Twenty One)"
ID=fedora
VERSION_ID=21
PRETTY_NAME="Fedora 21 (Twenty One)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:fedoraproject:fedora:21"
HOME_URL="https://fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=21
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=21

hongbin (hongbin034)
Changed in magnum:
status: New → Triaged
importance: Undecided → Medium
Changed in magnum:
assignee: nobody → Leticia Garcia Martin (laetitiae)
status: Triaged → In Progress
Changed in magnum:
assignee: Leticia Garcia Martin (laetitiae) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.