K8s-coredump handler: Coredump is not created when using capital G in "core_max_size: 5G" for pod mounted to a PVC

[Heron Vieira]

Brief Description
-----------------
It was noticed that for a pod mounted to PVC, pod coredump is not been generated if 'starlingx.io/core_max_size: 5G' is using uppercase.

Likely applied to another size annotations. This needs to be validated too.

Severity
--------
Minor

Steps to Reproduce
------------------
Create YAML definitions for PVC and POD:

// pvc-definition.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-claim1
spec:
  storageClassName: general
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

// pod (make sure core_max_size annotation is using capital G):

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu-pod-with-annotation
  annotations:
    starlingx.io/core_pattern: "/coredump-log/core.%P.%u.%g.%s.%t.%e"
    starlingx.io/core_compression: lz4
    starlingx.io/core_max_size: 5G
    starlingx.io/core_max_used: 50%
    starlingx.io/core_min_free: 20%
spec:
  containers:
  - name: ubuntu-pod-with-annotation
    image: ubuntu
    command: ["/bin/bash", "-ec", "while :; do echo '.'; sleep 5 ; done"]
    volumeMounts:
    - name: pvc
      mountPath: "/coredump-log"
  restartPolicy: "Never"
  volumes:
  - name: pvc
    persistentVolumeClaim:
      claimName: pvc-claim1

Deploy the PVC and POD

$ kubectl create -f pvc-definition.yaml
persistentvolumeclaim/pvc-claim1 created

$ kubectl create -f ubuntu-pod.yaml
pod/ubuntu-pod-with-annotation created

Connect to the pod to kill the application

Note: In our case, the pod was created to run sleep process:
command: ["/bin/bash", "-ec", "while :; do echo '.'; sleep 5 ; done"]

// Connect to the pod
$ kubectl exec -it ubuntu-pod-with-annotation -- /bin/sh

// Check the coredump-log folder has been created:
# ls -ltr | grep core
drwxr-xr-x 3 root root 4096 Sep 29 20:22 coredump-log

// Move to the directory
# cd coredump-log

// Kill the application running on pod
# kill -s 5 $(pgrep sleep)

// Pod will be in Error state:
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
ubuntu-pod-with-annotation 0/1 Error 0 5m4s

Redeploy the pod to check coredump

// First, delete the pod in Error state
$ kubectl delete pod ubuntu-pod-with-annotation

// Deploy again
$ kubectl create -f ubuntu-pod.yaml

// Connect to the pod and check coredump has been generated
$ kubectl exec -it ubuntu-pod-with-annotation -- /bin/sh

// See that no coredump has been generated
# ls coredump-log
lost+found
#

Expected Behavior
------------------
Coredump generated using either lowercase or uppercase for 'core_max_size' annotation.

Actual Behavior
----------------
Coredump is NOT generated when using uppercase G, e.g., 'starlingx.io/core_max_size: 5G'.

Reproducibility
---------------
Intermittent

System Configuration
--------------------
DX+

Timestamp/Logs
--------------
INFO:k8s-coredump:Getting annotations config
INFO:k8s-coredump:Annotations config: {'core_pattern': '/coredump-log/core.%P.%u.%g.%s.%t.%e', 'file_size_config': '5G', 'file_compression_config': 'lz4', 'max_use_config': '50%', 'keep_free_config': '20%'}
INFO:k8s-coredump:Pod default/ubuntu-pod-with-annotation handling core dump for 3758139
INFO:k8s-coredump:Parsing core pattern: /coredump-log/core.%p.%u.%g.%s.%t.%e
INFO:k8s-coredump:Core pattern parsed to /coredump-log/core.3758139.0.0.5.1696008518.sleep
DEBUG:k8s-coredump:podCoreFile: corefile=/coredump-log/core.3758139.0.0.5.1696008518.sleep
INFO:k8s-coredump:Starting to write coredump file /coredump-log/core.3758139.0.0.5.1696008518.sleep
DEBUG:nsenter:Entering mnt namespace 3758139
INFO:k8s-coredump:Checking available space on /coredump-log
INFO:k8s-coredump:Space info for /coredump-log: Total - 10464022528 bytes | Used - 24576 bytes (0.00 %) | Free - 10447220736 bytes (99.84 %)
DEBUG:nsenter:Leaving mnt namespace 3758139
INFO:k8s-coredump:Parsing size config: 5G
DEBUG:k8s-coredump:Pod does have annotations defined
INFO:k8s-coredump:No pod information was found, using default system coredump. Command: ['/usr/lib/systemd/systemd-coredump', '3758139', '0', '0', '5', '1696008518', '9223372036854775808', 'ubuntu-pod-with-annotation', 'sleep']
INFO:k8s-coredump:Dumped through default core process

Test Activity
-------------
Testing

Workaround
----------
Use lowercase 'g':

starlingx.io/core_max_size: 5g

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/utilities/+/902153
Committed: https://opendev.org/starlingx/utilities/commit/a9710363943c805f5f874bb3097386539d95e061
Submitter: "Zuul (22348)"
Branch: master

commit a9710363943c805f5f874bb3097386539d95e061
Author: Heron Vieira <email address hidden>
Date: Wed Nov 29 11:38:14 2023 -0300

    Improve logs and case-sensitive on k8s-coredump

    Improving logging by adding timestamp and
    show error in case we have a KeyError while
    trying to verify configurations set by
    pod annotations.
    Also allowing case-sensitive paths to be
    configured as the path to save the coredumps.

    PASS: AIO-SX install and bootstrap.
    PASS: On a AIO-SX test if k8s-coredump is executed
            successfully.
    PASS: On a AIO-DX test if k8s-coredump is executed
            successfully.
    PASS: On a Standard test if k8s-coredump is executed
            successfully.
    PASS: On a Standard test if k8s-coredump is executed
            successfully using a PVC setup.

    Partial-Bug: 2045789

    Change-Id: I942253468a596b7d68d6a3733b24062cfb4661f7
    Signed-off-by: Heron Vieira <email address hidden>