Hi,

This time, it appears to be because a "status" document doesn't have a "txn-revno" field :

juju:PRIMARY> db.statuses.find({ _id: { $regex: "ksplice/31#charm" } }).pretty()
{
        "_id" : "0b399859-692f-4e41-8f6d-c3a837cd26c3:u#ksplice/31#charm",
        "env-uuid" : "0b399859-692f-4e41-8f6d-c3a837cd26c3",
        "status" : "unknown",
        "statusdata" : {

        },
        "statusinfo" : "",
        "txn-queue" : [
                "5855f8181c56b63028334e84_66b7dbac",
                "58613aac1c56b670e01053dd_d6947399",
                "5865225a1c56b670e01bcf58_949b14d0",
                "5865e7ff1c56b670e01e1457_62b06131",
                "586b8f841c56b670e02eb517_35ffc19a"
        ]
}




Let's look at the txn-queue :



juju:PRIMARY> db.txns.find({ _id: ObjectId('5855f8181c56b63028334e84') }).pretty()
{
        "_id" : ObjectId("5855f8181c56b63028334e84"),
        "n" : "165ddd7e",
        "o" : [
                {
                        "c" : "statuses",
                        "d" : "0b399859-692f-4e41-8f6d-c3a837cd26c3:u#ksplice/31#charm",
                        "a" : {
                                "txn-revno" : NumberLong(0)
                        },
                        "u" : {
                                "$set" : {
                                        "env-uuid" : "0b399859-692f-4e41-8f6d-c3a837cd26c3",
                                        "status" : "active",
                                        "statusinfo" : "Effective kernel 3.13.0-105-generic",
                                        "statusdata" : {

                                        },
                                        "updated" : NumberLong("1482029080376315530"),
                                        "neverset" : false
                                }
                        }
                }
        ],
        "s" : 5
}



As you can see, the only assertion is that txn-revno == 0, which I guess isn't the case because there no "txn-revno" field ? This causes the state to be set to 5 ("aborted"). And causes the status update to fail.

The other queued txn are similar, e.g. :

juju:PRIMARY> db.txns.find({ _id: ObjectId('58613aac1c56b670e01053dd') }).pretty()
{
        "_id" : ObjectId("58613aac1c56b670e01053dd"),
        "n" : "189c1ad6",
        "o" : [
                {
                        "c" : "statuses",
                        "d" : "0b399859-692f-4e41-8f6d-c3a837cd26c3:u#ksplice/31#charm",
                        "a" : {
                                "txn-revno" : NumberLong(0)
                        },
                        "u" : {
                                "$set" : {
                                        "env-uuid" : "0b399859-692f-4e41-8f6d-c3a837cd26c3",
                                        "status" : "active",
                                        "statusinfo" : "Effective kernel 3.13.0-106-generic",
                                        "statusdata" : {

                                        },
                                        "updated" : NumberLong("1482767020431622924"),
                                        "neverset" : false
                                }
                        }
                }
        ],
        "s" : 5
}



Example of a status document which works - you can see the txn-revno field :


juju:PRIMARY> db.statuses.find({ _id: { $regex: "ksplice/45#charm" } }).pretty()
{
        "_id" : "0b399859-692f-4e41-8f6d-c3a837cd26c3:u#ksplice/45#charm",
        "env-uuid" : "0b399859-692f-4e41-8f6d-c3a837cd26c3",
        "neverset" : false,
        "status" : "active",
        "statusdata" : {

        },
        "statusinfo" : "Effective kernel 3.13.0-106-generic",
        "txn-queue" : [
                "586f68f61c56b6227e004006_17264e09"
        ],
        "txn-revno" : NumberLong(6194),
        "updated" : NumberLong("1483696374166671919")
}



What should we do here ?

Thanks