watch can fail (hang) under python3

Bug #1950517 reported by Tyler Stachecki
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
etcd3gw
Fix Released
Undecided
Tyler Stachecki

Bug Description

Looking here:
https://opendev.org/openstack/etcd3gw/src/commit/9d50411b6ff5d34f1d1036dbe344ca3e8c4cae7e/etcd3gw/watch.py#L22

The iter_content method is requested to do decode of the string as needed, from bytes to unicode. On the very next line, an unconditional decode of the string is done (again).

While this is fine in python2:
Python 2.7.16 (default, Mar 25 2021, 03:11:28)
[GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1103.0.29.20) (-macos10.15-objc- on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> b'bytes'.decode('utf-8').decode('utf-8')
u'bytes'

It causes an AttributeError in python3:
Python 3.8.8 (v3.8.8:024d8058b0, Feb 19 2021, 08:48:17)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> b'bytes'.decode('utf-8').decode('utf-8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute 'decode'

This can be hard to see in client code, as etcd3gw.Watcher does not call result() on the future, and so the exception is never materialized.

I am opening this LP so I can reference a fix in the review system.

Changed in python-etcd3gw:
assignee: nobody → Tyler Stachecki (tstachecki)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to etcd3gw (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/etcd3gw/+/817489

Changed in python-etcd3gw:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to etcd3gw (master)

Reviewed: https://review.opendev.org/c/openstack/etcd3gw/+/817489
Committed: https://opendev.org/openstack/etcd3gw/commit/ed899b34e464862525f76fff2377a2cceeb65af7
Submitter: "Zuul (22348)"
Branch: master

commit ed899b34e464862525f76fff2377a2cceeb65af7
Author: Tyler J. Stachecki <email address hidden>
Date: Wed Nov 10 17:32:03 2021 -0500

    watch: Avoid double-decoding in python3

    Both iter_content and the first line of the loop in the
    etcd3gw watch code were decoding the chunk received.

    In python2, this works fine:
    ```
    $ python2 -c "print(b'hello_world'.decode().decode('utf-8'))"
    hello_world
    ```

    In python3, it raises an AttributeError:
    ```
    $ python3 -c "print(b'hello_world'.decode().decode('utf-8'))"
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    AttributeError: 'str' object has no attribute 'decode'
    ```

    Thus, etcd3gw's watch is broken in python3. This commit
    fixes it by only decoding the line once.

    Signed-off-by: Tyler J. Stachecki <email address hidden>
    Change-Id: I203574a1ef4996a43860350be59fbe208457562d
    Closes-Bug: 1950517

Changed in python-etcd3gw:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/etcd3gw 1.0.1

This issue was fixed in the openstack/etcd3gw 1.0.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.