Comment 1 for bug 1776673

Note that another possibility would be to update the txn from status "2" ==
"prepared" to status "1" which is "preparing", in which case it should try
to add the txn id to all the queues. I think you also need to delete the
"n" field. So something like:

db.txns.update({_id: " 5b1f25665f5ce833efeff19b"}, {$set: {"s": 1}, $unset:
{"n": 1}})

On Wed, Jun 13, 2018 at 5:21 PM, Junien Fridrick <<email address hidden>
> wrote:

> Public bug reported:
>
> Hi,
>
> Last night, we suddenly got the following logs on our Juju 2.3.7
> controller :
>
> 2018-06-12 01:45:23 ERROR juju.state cleanup.go:150 cleanup failed for
> units("ubuntu"): cannot destroy unit "ubuntu/37264": cannot find
> transaction 5b1f25665f5ce833efeff19b_0e03bf64 in queue for document
> {annotations 6cd2c273-658c-44c9-8cff-42f47eb586ea:a#ubuntu}
>
> I have no idea how it happened. There was some load on the mongodb
> primary, presumably because we were in the process of backing it up
> (mongodump).
>
> Also, this happened during an internal prune :
>
> 2018-06-12 01:43:50 INFO juju.txn prune.go:127 txns after last prune:
> 209061, txns now: 356448, pruning: too many new transactions
>
> 2018-06-12 01:49:19 INFO juju.txn prune.go:152 txn pruning complete after
> 5m29.68502757s. txns now: 168206, inspected 68 collections, 24978 docs (623
> cleaned)
> removed 334 stash docs and 208935 txn docs
>
>
> The txn was indeed the first in the queue for the unit. However, it wasn't
> in the queue for the annotation document (which was already in txns.stash).
> jam advised to try to add it, which I did, using :
>
> db.txns.stash.update(
>
> {"_id.id": "6cd2c273-658c-44c9-8cff-42f47eb586ea:a#ubuntu"},
> {$push: {
> "txn-queue": {
> $each: ["5b1f25665f5ce833efeff19b_0e03bf64"],
> $position: 0
> }
> }}
> )
>
> (with the controllers down). After that I ran an mgopurge, and the
> problem went away.
>
> ** Affects: juju
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1776673
>
> Title:
> txn suddenly went missing in a txn queue
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1776673/+subscriptions
>