ValueError when running manila list --count True"

[Eric Harney]

ValueError when running manila list --count True"

This happens because cs.shares.list can return different data types depending on what's requested and what shares exist.

https://git.openstack.org/cgit/openstack/python-manilaclient/tree/manilaclient/v2/shell.py?h=1.27.0#n1932

This code makes an assumption that if "with_count" is specified, cs.shares.list will return a tuple of (shares, total_count). But if there are no shares, it only returns [].

manilaclient @ 6c11461

$ manila list
+----+------+------+-------------+--------+-----------+-----------------+------+-------------------+
| ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone |
+----+------+------+-------------+--------+-----------+-----------------+------+-------------------+
+----+------+------+-------------+--------+-----------+-----------------+------+-------------------+

$ manila list --count True
ERROR: need more than 0 values to unpack

$ manila --debug list --count True
<snip>
DEBUG (httpclient:199)
REQ: curl -i -X GET http://10.13.57.164:8786/v2/ceca990548ad4a6d8ad9f76908e84e85/shares/detail?with_count=True -H "X-Openstack-Manila-Api-Version: 2.48" -H "X-Auth-Token: gAAAAABco3QH7faxc17IjDTaxa9_tH3VkYWuzgKG_2bBQjxzV5x0qU--GrJ1hCiZ6ATP9J6fTN60FGfK_nzDa_lw7HiuMItsX2pUFjJA1_UXUnfnAyYdoRi0_ZPU6ofNd9_qjyVWi3x8BfRcOop2H3Smy4xGvhqD0CObhv_QRGWGRWkGUCKosMM" -H "Accept: application/json" -H "User-Agent: python-manilaclient"

RESP: [200] {'content-length': '14', 'x-compute-request-id': 'req-960abfe9-c83a-469d-8882-c145cf850f94', 'vary': 'X-OpenStack-Manila-API-Version', 'connection': 'close', 'server': 'Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_wsgi/3.4 Python/2.7.5', 'x-openstack-manila-api-version': '2.48', 'date': 'Tue, 02 Apr 2019 14:39:03 GMT', 'content-type': 'application/json'}
RESP BODY: {"shares": []}

DEBUG (httpclient:208) RESP: [200] {'content-length': '14', 'x-compute-request-id': 'req-960abfe9-c83a-469d-8882-c145cf850f94', 'vary': 'X-OpenStack-Manila-API-Version', 'connection': 'close', 'server': 'Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_wsgi/3.4 Python/2.7.5', 'x-openstack-manila-api-version': '2.48', 'date': 'Tue, 02 Apr 2019 14:39:03 GMT', 'content-type': 'application/json'}
RESP BODY: {"shares": []}

DEBUG (shell:707) need more than 0 values to unpack
Traceback (most recent call last):
  File "/opt/stack/python-manilaclient/manilaclient/shell.py", line 702, in main
    map(encodeutils.safe_decode, sys.argv[1:]))
  File "/opt/stack/python-manilaclient/manilaclient/shell.py", line 571, in main
    args.func(self.cs, args)
  File "/opt/stack/python-manilaclient/manilaclient/v2/shell.py", line 1936, in do_list
    sort_dir=args.sort_dir,
ValueError: need more than 0 values to unpack
ERROR: need more than 0 values to unpack

[Tom Barron (tpb)]

maybe we can do something along these lines ...

diff --git a/manilaclient/v2/shell.py b/manilaclient/v2/shell.py
index 0a5646c..1a3c20d 100644
--- a/manilaclient/v2/shell.py
+++ b/manilaclient/v2/shell.py
@@ -1931,10 +1931,13 @@ def do_list(cs, args):
     total_count = 0
     if strutils.bool_from_string(args.count, strict=True):
         search_opts['with_count'] = args.count
- shares, total_count = cs.shares.list(
- search_opts=search_opts, sort_key=args.sort_key,
- sort_dir=args.sort_dir,
- )
+ try:
+ shares, total_count = cs.shares.list(
+ search_opts=search_opts, sort_key=args.sort_key,
+ sort_dir=args.sort_dir,
+ )
+ except ValueError:
+ shares, total_count = [], 0
     else:
         shares = cs.shares.list(
             search_opts=search_opts, sort_key=args.sort_key,

[Goutham Pacha Ravi (gouthamr)]

I commented on the patch Soledad submitted, but I'll add the relevant notes here:

I don't believe this bug originates in the client; The expectation when the API parameter "with_count" is set to True is that the server must *always* return the count of the records retrieved. It's true, we can defensively code a fix in the client to workaround issues when the server doesn't return a count... but, we wouldn't fix the actual problem if we stopped at that.

Our problem is here: https://github.com/openstack/manila/blob/cec13b8057fa025fa1b1c3bef180687a394d6fd6/manila/api/views/shares.py#L181

When count is 0, the if-condition fails - the condition should have been: "if count is not None" rather than just "if count".

[OpenStack Infra (hudson-openstack)]

Change abandoned by Soledad Kuczala (<email address hidden>) on branch: master
Review: https://review.opendev.org/667744
Reason: The bug is in manila API

[Goutham Pacha Ravi (gouthamr)]

Manila patch is here: https://review.opendev.org/#/c/673913/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/673913
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=60254a4fd2bcb619ccf394e143583c6c226a21b9
Submitter: Zuul
Branch: master

commit 60254a4fd2bcb619ccf394e143583c6c226a21b9
Author: SolKuczala <email address hidden>
Date: Wed Jul 31 20:47:36 2019 +0000

    Fix _list_view function for count

    When doing `manila list --count True` without shares, the user gets
    an error. This happens because _list_view do not handle count
    properly for 0 shares. This patch-set adds a fix for this case.

    Change-Id: Ic6b45260ae39da9ec2c29d05c76d85be1e20635d
    Closes-bug: #1822815

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 9.0.0.0rc1 release candidate.