Unexpected get_clk_class behaviour when using silicom STS2

[Caio Bruchert]

Brief Description

 when we  apply Synce services and trying to check Clock Class using "get_clk_class" it need to return "Clock Class: 6, LOCKED"  but we are getting "Clock Class: 7, LOCKED"
Severity

Standard

Steps to Reproduce

Create sts_override.yaml

[sysadmin@controller-0 ~(keystone_admin)]$ cat > sts_override_T-GM_synce.yaml << EOF
Spec:
  profileID: 2
  ports:
  - ethName: enp81s0f1
    ql: 4
    ethPort: 4
  - ethName: enp81s0f3
    ql: 4
    ethPort: 3
  masterPortMask_GM: 0xfff
  syncePortMask_GM: 0xfff
  twoStep: 1
  domainNum_8275_1: 24
EOF

Upload sts-silicom

[sysadmin@controller-0 ~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/sts-silicom-*.tgz
[sysadmin@controller-0 ~(keystone_admin)]$ system helm-override-update sts-silicom sts-silicom sts-silicom --values sts_override_T-GM_synce.yaml
[sysadmin@controller-0 ~(keystone_admin)]$ system application-apply sts-silicom
[sysadmin@controller-0 ~(keystone_admin)]$ kubectl get pods -n sts-silicom -o=wide
NAME                    READY   STATUS    RESTARTS   AGE   IP                       NODE           NOMINATED NODE   READINESS GATES
sts-silicom-tsy-5hnv7   4/4     Running   0          58s   fdff:719a:bf60:1009::3   controller-0   <none>           <none>

go into app:

[sysadmin@controller-0 ~(keystone_admin)]$ kubectl exec -it sts-silicom-tsy-5hnv7 -c sts-silicom-grp -n sts-silicom -- tsynctl_grpc

Tsynctl gRPC Client v1.1.3

$ get_mode
Mode: 2, PTP Master Mode
$ get_clk_class
Clock Class: 6, LOCKED

Expected Behavior

$ get_clk_class
Clock Class: 6, LOCKED

Actual Behavior

$ get_clk_class
Clock Class: 7, LOCKED

Reproducibility

out of 5 time its failing around 3 time, its like a intermediate issue.

System Configuration

System Type: simplex

Load info (eg: 2022-03-10_20-00-07)
master

Last Pass

NA

Alarms

No alarms found on the system.

Test Activity

Automated Run: Regression

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/app-sts-silicom/+/901620

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/config/+/901623

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/app-sts-silicom/+/901620
Committed: https://opendev.org/starlingx/app-sts-silicom/commit/520eee97f63d5971ea8f7ec2891f1c04dadbb6c3
Submitter: "Zuul (22348)"
Branch: master

commit 520eee97f63d5971ea8f7ec2891f1c04dadbb6c3
Author: Caio Bruchert <email address hidden>
Date: Tue Nov 21 16:26:51 2023 -0300

    Fix PTP sts-silicom app locking issue

    STS app was intermittently returning "Clock Class: 7, LOCKED" instead
    of "Clock Class: 6, LOCKED" or not locking at all with tsyncd getting
    stuck.

    This commit upgrades the app's images to fix this issue.
    The following images releases are being used now:

    Images:
      Tsyncd: quay.io/silicom/tsyncd:2.1.3.6
      Phc2Sys: quay.io/silicom/phc2sys:3.1-00193-g6bac465
      GrpcTsyncd: quay.io/silicom/grpc-tsyncd:2.1.2.18
      Gpsd: quay.io/silicom/gpsd:3.23.1

    Test Plan:
    PASS: Apply sts-silicom app and check that it gets locked to
          clock class 6
    PASS: Delete sts-silicom pod several times and check that it
          always gets locked to clock class 6
    PASS: Lock/unlock the controller several times and check that
          the STS app always gets locked to clock class 6

    Closes-Bug: 2044178

    Change-Id: I234e7a6d7912ce70c90ffc4a81059ca0d7346499
    Signed-off-by: Caio Bruchert <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/901623
Committed: https://opendev.org/starlingx/config/commit/64e1b35522a534fbcb47293fb3e28e4a42c65b4a
Submitter: "Zuul (22348)"
Branch: master

commit 64e1b35522a534fbcb47293fb3e28e4a42c65b4a
Author: Caio Bruchert <email address hidden>
Date: Tue Nov 21 17:00:12 2023 -0300

    Upgrade sts-silicom app

    Images:
      Tsyncd: quay.io/silicom/tsyncd:2.1.3.6
      Phc2Sys: quay.io/silicom/phc2sys:3.1-00193-g6bac465
      GrpcTsyncd: quay.io/silicom/grpc-tsyncd:2.1.2.18
      Gpsd: quay.io/silicom/gpsd:3.23.1

    Test Plan:
    PASS: Apply sts-silicom app and check that it gets locked to
          clock class 6
    PASS: Delete sts-silicom pod several times and check that it
          always gets locked to clock class 6
    PASS: Lock/unlock the controller several times and check that
          the STS app always gets locked to clock class 6

    Closes-Bug: 2044178
    Depends-On: https://review.opendev.org/c/starlingx/app-sts-silicom/+/901620

    Signed-off-by: Caio Bruchert <email address hidden>
    Change-Id: I7699c41e54fc2247295b6bb526f506c64c81a389

[Ghada Khalil (gkhalil)]

Re-opening as the sts-silicom app appears to be having issues w/ k8s 1.26, so the original fix is not functional

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/app-sts-silicom/+/903787

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/app-sts-silicom/+/903787
Committed: https://opendev.org/starlingx/app-sts-silicom/commit/b885b3e40051abfdbd91eab208a2292cb37ebad3
Submitter: "Zuul (22348)"
Branch: master

commit b885b3e40051abfdbd91eab208a2292cb37ebad3
Author: Caio Bruchert <email address hidden>
Date: Fri Dec 15 16:12:35 2023 -0300

    sts-silicom: disable PodSecurityPolicy

    After k8s upgrade and the PodSecurityPolicy removal,
    sts-silicom app stopped working (system application-apply fails).

    This fix disables PSP until Silicom releases a fix for it.

    Test Plan:
        PASS: upload sts-silicom app
        PASS: apply sts-silicom app
        PASS: check that to pod is running
        PASS: check that there are no errors in syinv.log
        PASS: check if clock class is locked using tsynctl_grpc

    Closes-Bug: 2044178

    Change-Id: Iccb95ea62e2eb357f0c5806560e16bb81c09c4a7
    Signed-off-by: Caio Bruchert <email address hidden>