Juju container controller MongoDB startup script suppresses all mongod output

Bug #1982832 reported by Weii Wang
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fix Released
Weii Wang

Bug Description

One of the containers inside juju k8s controller pod is juju-db which starts a mongodb instance. The current startup script for that container is using bash command substitution to launch the mongodb process.


$(mongod ${args})

When the mongod failed to start, or some error occurred causing mongod to exit, command substitution will cause the shell to treat everything printed by mongod as a new command and try to execute that command. Of course, that’s not a command, so the only message left was some gibberish produced by shell when the shell is trying to run mongod’s output as a command. This will cause all the valuable debug information about the mongod to be lost.

The log remained will look like this:
/root/mongo.sh: 6: {"t":{"$date":"2022-07-14T06:18:39.191Z"},"s":"W",: not found

To reproduce this error, first bootstrap juju in k8s, then find the juju controller namespace and pod name:

$ juju version

$ kubectl get namespaces \
    -o=custom-columns=:.metadata.name | grep controller

$ kubectl get pods -n controller-test \
    -o=custom-columns=:.metadata.name | grep controller

Then manually send SIGTERM signal to mongod:

$ kubectl exec controller-0 -n controller-test \
    --container mongodb -- bash -c "pkill mongod"

Watch the log of the “failed” container:

$ kubectl logs controller-0 -n controller-test \
    --container mongodb -p
/root/mongo.sh: 6: {"t":{"$date":"2022-07-14T06:18:39.191Z"},"s":"W",: not found

Weii Wang (weii-wang)
description: updated
Revision history for this message
Weii Wang (weii-wang) wrote :

I have proposed a simple patch for this issue.

Ian Booth (wallyworld)
Changed in juju:
milestone: none → 2.9.33
assignee: nobody → Weii Wang (weii-wang)
importance: Undecided → Medium
status: New → In Progress
Ian Booth (wallyworld)
Changed in juju:
status: In Progress → Fix Committed
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers