Configuring k8s-dashboard using exposed ApiServer did not work as k8s documentation describes

Bug #1838734 reported by Pedro Guimarães
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
CDK Addons
Triaged
Medium
Unassigned
Kubernetes Control Plane Charm
Triaged
Medium
Unassigned

Bug Description

I am currently trying to configure kubernetes-dashboard as described on: https://github.com/kubernetes/dashboard/wiki/Accessing-Dashboard---1.7.X-and-above#api-server

I currently have a cluster with exposed API Server via Kubeapi-LB VIP to 10.66.0.20.

Bug related to:
https://bugs.launchpad.net/charm-kubernetes-master/+bug/1835396

$ kubectl get svc --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 11h
kube-system heapster ClusterIP 10.152.183.168 <none> 80/TCP 11h
kube-system kube-controller-manager-prometheus-discovery ClusterIP None <none> 10252/TCP 3h39m
kube-system kube-dns ClusterIP 10.152.183.240 <none> 53/UDP,53/TCP,9153/TCP 11h
kube-system kube-scheduler-prometheus-discovery ClusterIP None <none> 10251/TCP 3h39m
kube-system kubelet ClusterIP None <none> 10250/TCP 3h35m
kube-system kubernetes-dashboard ClusterIP 10.152.183.62 <none> 443/TCP 11h
kube-system metrics-server ClusterIP 10.152.183.19 <none> 443/TCP 11h
kube-system monitoring-grafana ClusterIP 10.152.183.129 <none> 80/TCP 11h
kube-system monitoring-influxdb ClusterIP 10.152.183.7 <none> 8083/TCP,8086/TCP 11h
monitoring alertmanager-main ClusterIP 10.152.183.8 <none> 9093/TCP 3h39m
monitoring alertmanager-operated ClusterIP None <none> 9093/TCP,6783/TCP 3h39m
monitoring grafana ClusterIP 10.152.183.9 <none> 3000/TCP 3h39m
monitoring kube-state-metrics ClusterIP None <none> 8443/TCP,9443/TCP 3h39m
monitoring node-exporter ClusterIP None <none> 9100/TCP 3h39m
monitoring prometheus-k8s ClusterIP 10.152.183.42 <none> 9090/TCP 3h39m
monitoring prometheus-operated ClusterIP None <none> 9090/TCP 3h38m
monitoring prometheus-operator ClusterIP None <none> 8080/TCP 3h39m

Trying:
$ wget https://10.152.183.62:443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ --no-check-certificate
--2019-08-02 07:20:32-- https://10.152.183.62/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/
Connecting to 10.152.183.62:443... connected.
WARNING: cannot verify 10.152.183.62's certificate, issued by ‘CN=.’:
  Self-signed certificate encountered.
    WARNING: certificate common name ‘.’ doesn't match requested host name ‘10.152.183.62’.
HTTP request sent, awaiting response... 404 Not Found
2019-08-02 07:20:32 ERROR 404: Not Found.

How can I expose kubernetes-dashboard in this case?

Revision history for this message
George Kraft (cynerva) wrote :

The correct URL should have the IP of the Kubernetes API, not the IP of the kubernetes-dashboard service. Try this: https://10.66.0.20:443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

Alternatively, you can use `kubectl proxy` as covered in our docs on this page: https://ubuntu.com/kubernetes/docs/operations

Changed in charm-kubernetes-master:
status: New → Incomplete
Revision history for this message
Pedro Guimarães (pguimaraes) wrote :

When trying to reach out for URL with API Server VIP, I get:

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "services \"https:kubernetes-dashboard:\" is forbidden: User \"system:anonymous\" cannot get resource \"services/proxy\" in API group \"\" in the namespace \"kube-system\"",
  "reason": "Forbidden",
  "details": {
    "name": "https:kubernetes-dashboard:",
    "kind": "services"
  },
  "code": 403
}

I am not sure if I should give system:anonymous access to services/proxy. I would rather prefer to use kubectl proxy and have my user authenticated beforehand.

Revision history for this message
Peter Sabaini (peter-sabaini) wrote :

I'm also seeing this priv fault. Giving an anonymous user access seems risky, is there an alternative option?

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "services \"https:kubernetes-dashboard:\" is forbidden: User \"system:anonymous\" cannot get resource \"services/proxy\" in API group \"\" in the namespace \"kube-system\"",
  "reason": "Forbidden",
  "details": {
    "name": "https:kubernetes-dashboard:",
    "kind": "services"
  },
  "code": 403
}

Changed in charm-kubernetes-master:
status: Incomplete → New
Revision history for this message
George Kraft (cynerva) wrote :

Currently, the only way we have documented to access the dashboard is via `kubectl proxy` as seen on this page: https://ubuntu.com/kubernetes/docs/operations

I think what's missing in the charms is the ability to specify a service type for the kubernetes-dashboard service. We should allow the operator to expose the dashboard as a NodePort or LoadBalancer service for external accessibility.

Changed in charm-kubernetes-master:
importance: Undecided → Medium
Changed in cdk-addons:
importance: Undecided → Medium
Changed in charm-kubernetes-master:
status: New → Triaged
Changed in cdk-addons:
status: New → Triaged
Revision history for this message
George Kraft (cynerva) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.