Comment 16 for bug 1788722

Revision history for this message
Bruno Carvalho (brunowcs) wrote :

This problem can be related my crushmap:

ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 82.63184 root default
-2 65.47192 rack sata
-4 32.73596 host uat-l-stor-11
 0 5.45599 osd.0 up 1.00000 1.00000
 1 5.45599 osd.1 up 1.00000 1.00000
 2 5.45599 osd.2 up 1.00000 1.00000
 3 5.45599 osd.3 up 1.00000 1.00000
 4 5.45599 osd.4 up 1.00000 1.00000
 5 5.45599 osd.5 up 1.00000 1.00000
-5 32.73596 host uat-l-stor-12
 6 5.45599 osd.6 up 1.00000 1.00000
 7 5.45599 osd.7 up 1.00000 1.00000
 8 5.45599 osd.8 up 1.00000 1.00000
 9 5.45599 osd.9 up 1.00000 1.00000
10 5.45599 osd.10 up 1.00000 1.00000
11 5.45599 osd.11 up 1.00000 1.00000
-3 17.15991 rack ssd
-6 8.57996 host uat-l-stor-13
12 1.42999 osd.12 up 1.00000 1.00000
13 1.42999 osd.13 up 1.00000 1.00000
14 1.42999 osd.14 up 1.00000 1.00000
15 1.42999 osd.15 up 1.00000 1.00000
16 1.42999 osd.16 up 1.00000 1.00000
17 1.42999 osd.17 up 1.00000 1.00000
-7 8.57996 host uat-l-stor-14
18 1.42999 osd.18 up 1.00000 1.00000
19 1.42999 osd.19 up 1.00000 1.00000
20 1.42999 osd.20 up 1.00000 1.00000
21 1.42999 osd.21 up 1.00000 1.00000
22 1.42999 osd.22 up 1.00000 1.00000
23 1.42999 osd.23 up 1.00000 1.00000

I believe my mistake lies in this function:

def get_upgrade_position(osd_sorted_list, match_name):
    """Return the upgrade position for the given osd.

    :param osd_sorted_list: list. Osds sorted
    :param match_name: str. The osd name to match
    :returns: int. The position or None if not found
    """
    for index, item in enumerate(osd_sorted_list):
        log('index %s' % index ,DEBUG)
        log('item.name %s' % item.name ,DEBUG)
        log('match_name %s' % match_name ,DEBUG)

        if item.name == match_name:
            log('pass in if %s' % index, DEBUG)
            return index
    return None

See the logs below if it is never true is function return None "DEBUG juju-log upgrade position None"

2018-10-09 23:41:29 DEBUG worker.uniter.jujuc server.go:178 running hook tool "juju-log"
2018-10-09 23:41:29 DEBUG juju-log index 0
2018-10-09 23:41:29 DEBUG worker.uniter.jujuc server.go:178 running hook tool "juju-log"
2018-10-09 23:41:29 DEBUG juju-log item.name sata
2018-10-09 23:41:29 DEBUG worker.uniter.jujuc server.go:178 running hook tool "juju-log"
2018-10-09 23:41:29 DEBUG juju-log match_name uat-l-stor-11
2018-10-09 23:41:30 DEBUG worker.uniter.jujuc server.go:178 running hook tool "juju-log"
2018-10-09 23:41:30 DEBUG juju-log index 1
2018-10-09 23:41:30 DEBUG worker.uniter.jujuc server.go:178 running hook tool "juju-log"
2018-10-09 23:41:30 DEBUG juju-log item.name ssd
2018-10-09 23:41:30 DEBUG worker.uniter.jujuc server.go:178 running hook tool "juju-log"
2018-10-09 23:41:30 DEBUG juju-log match_name uat-l-stor-11
2018-10-09 23:41:30 DEBUG worker.uniter.jujuc server.go:178 running hook tool "juju-log"
2018-10-09 23:41:30 DEBUG juju-log upgrade position None
2018-10-09 23:41:30 DEBUG config-changed Traceback (most recent call last):
2018-10-09 23:41:30 DEBUG config-changed File "/var/lib/juju/agents/unit-ceph-osd-3/charm/hooks/config-changed", line 704, in <module>
2018-10-09 23:41:30 DEBUG config-changed hooks.execute(sys.argv)
2018-10-09 23:41:30 DEBUG config-changed File "/var/lib/juju/agents/unit-ceph-osd-3/charm/hooks/charmhelpers/core/hookenv.py", line 847, in execute
2018-10-09 23:41:30 DEBUG config-changed self._hooks[hook_name]()
2018-10-09 23:41:30 DEBUG config-changed File "/var/lib/juju/agents/unit-ceph-osd-3/charm/hooks/charmhelpers/contrib/hardening/harden.py", line 79, in _harden_inner2
2018-10-09 23:41:30 DEBUG config-changed return f(*args, **kwargs)
2018-10-09 23:41:30 DEBUG config-changed File "/var/lib/juju/agents/unit-ceph-osd-3/charm/hooks/config-changed", line 408, in config_changed
2018-10-09 23:41:30 DEBUG config-changed check_for_upgrade()
2018-10-09 23:41:30 DEBUG config-changed File "/var/lib/juju/agents/unit-ceph-osd-3/charm/hooks/config-changed", line 137, in check_for_upgrade
2018-10-09 23:41:30 DEBUG config-changed upgrade_key='osd-upgrade')
2018-10-09 23:41:30 DEBUG config-changed File "lib/ceph/utils.py", line 2270, in roll_osd_cluster
2018-10-09 23:41:30 DEBUG config-changed osd_sorted_list[position -1].name))
2018-10-09 23:41:30 DEBUG config-changed TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

Someone may my help with this problem, this is a bug in charm with my crushmap?