Hplip does not work with Python3: Unicode error

Bug #1480152 reported by Johan Myréen on 2015-07-31
28
This bug affects 5 people
Affects Status Importance Assigned to Milestone
HPLIP
Undecided
Unassigned
hplip (Arch Linux)
New
Undecided
Unassigned
hplip (Debian)
Fix Released
Unknown
hplip (Fedora)
Confirmed
Undecided
hplip (Gentoo Linux)
Fix Released
Medium

Bug Description

The hp-setup program crashes with a UnicodeDecodeError when it tries to detect my Photosmart 2575 printer. I am running Arch Linux which has switched to using Python3 by default (/usr/bin/python is a symlink to python3).

Below is a part of a debugging session demonstrating the problem. The variable x contains (part of) the response from my printer to the multicast query sent on the network. The to_string_utf8 function is supposed to convert the contents of the byte array to UTF-8, but chokes on the 0xf0 in position 364, presumably because the bytes aren't all strictly ASCII codepoints.

Running the program using Python2 does not trigger this bug.

My printer is a HP Photosmart 2575, Python version 3.4.3, Python2 version 2.7.10, Hplip version hplip 3.15.6-1 (Arch Linux).

(Pdb) n
> /usr/share/hplip/base/slp.py(110)detectNetworkDevices()
-> struct.unpack("!BBHBBHHHHH", data[:16])
(Pdb) n
> /usr/share/hplip/base/slp.py(112)detectNetworkDevices()
-> x = struct.unpack("!%ds" % attr_length, data[16:])[0].strip()
(Pdb) n
> /usr/share/hplip/base/slp.py(115)detectNetworkDevices()
-> x= to_string_utf8(x)
(Pdb) p x
b'(x-hp-ver=01)(x-hp-mac=00187100690d)(x-hp-num_port=01)(x-hp-ip=192.168.000.102)(x-hp-hn=Smarty)(x-hp-p1=MFG:HP;MDL:Photosmart 2570 series;CMD:MLC,PCL,PML,DW-PCL,DESKJET,DYN;1284.4DL:4d,4e,1;CLS:PRINTER;DES:6543;SN:MY66U1131C04DY;S:038000C484001021002c1800028c2880025;J: ;Z:0102,0503dea9015549,0600;BT:000000000000,50686F746F736D6172742032353\xd1\xbdZT\xf0!\x80i \x82\xc8,\xff\xff\xfa@\xf0,\x85-)(x-hp-guid=00187100690d)'
(Pdb) n
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 364: invalid continuation byte
> /usr/share/hplip/base/slp.py(115)detectNetworkDevices()
-> x= to_string_utf8(x)
(Pdb)

Gaurav Sood (gaurav-sood) wrote :

Please apply this patch and see if it works
1. Take a back up of /usr/share/hplip/base/sixext.py
1. Download this patch in any folder and navigate to that folder
2. Apply this patch as a root user using the following command

    'patch /usr/share/hplip/base/sixext.py < sixext.patch'

Johan Myréen (r-ubuntuone-v) wrote :

Yes, now the decode functions ignore the errors and the setup succeeds.

Thank you.

Georges Toth (georges-toth) wrote :

The latest Debian Stretch version is affected by this bug, which is version 3.16.11+repack0-2.

I ended up patching the "/usr/share/hplip/base/sixext.py" file exactly as in the suggested patch, which solved the issue for us.

Is there any information on when this quite old fix will make it into the upstream package?

Changed in hplip (Debian):
importance: Undecided → Unknown
status: New → Unknown
Changed in hplip (Debian):
status: Unknown → Confirmed
kritika arora (kritika-arora1) wrote :

Hi,

The required changes has been included in HPLIP source and will be made available in the next release.

Thanks.

Changed in hplip:
status: New → Fix Committed
Georges Toth (georges-toth) wrote :

Hi,

Thank you!

Changed in hplip (Debian):
status: Confirmed → Fix Committed
Changed in hplip (Debian):
status: Fix Committed → Fix Released

According to #4 this has been fixed, however the same issue came up in Gentoo recently. Applying the patch from #1 fixed the issue.

What could be the problem here. Was this fix reverted or changed in another way. Maybe there is a regression somewhere.

Changed in hplip (Gentoo Linux):
importance: Unknown → Medium
status: Unknown → New
Changed in hplip (Gentoo Linux):
status: New → Fix Released
Changed in hplip:
status: Fix Committed → New

Tested version 3.18.4 on Arch, the problem is still there. The proposed patch has not been applied :

% hp-probe -bnet
HP Linux Imaging and Printing System (ver. 3.18.4)
(...)
\Traceback (most recent call last):
  File "/usr/bin/hp-probe", line 162, in <module>
    devices = device.probeDevices([b], timeout, ttl, filter_dict, search, method)
  File "/usr/share/hplip/base/device.py", line 487, in probeDevices
    detected_devices = slp.detectNetworkDevices(ttl, timeout)
  File "/usr/share/hplip/base/slp.py", line 115, in detectNetworkDevices
    x= to_string_utf8(x)
  File "/usr/share/hplip/base/sixext.py", line 113, in to_string_utf8
    return s.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 365-366: invalid continuation byte

The printer is an HP Photosmart C6480 All in One.

Download full text (5.0 KiB)

Description of problem:
Try to detect my network printer HP Photosmart C8260 using the command :
 % hp-probe -bnet -ldebug

Output of the command :

HP Linux Imaging and Printing System (ver. 3.18.4)
Printer Discovery Utility ver. 4.1

Copyright (c) 2001-15 HP Development Company, LP
This software comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to distribute it
under certain conditions. See COPYING file for more details.

--------------------
| DEVICE DISCOVERY |
--------------------

Probing network for printers. Please wait, this will take approx. 10 seconds...

hp-probe[1859]: debug: {}
hp-probe[1859]: debug: 0000: 01 07 01 a7 00 00 65 6e 00 03 00 00 00 00 01 97 28 78 2d 68 70 2d 76 65 72 3d 30 31 29 28 78 2d ......en........(x-hp-ver=01)(x-
hp-probe[1859]: debug: 0020: 68 70 2d 6d 61 63 3d 30 30 31 65 30 62 39 61 35 63 63 63 29 28 78 2d 68 70 2d 6e 75 6d 5f 70 6f hp-mac=001e0b9a5ccc)(x-hp-num_po
hp-probe[1859]: debug: 0040: 72 74 3d 30 31 29 28 78 2d 68 70 2d 69 70 3d 31 39 32 2e 31 36 38 2e 32 34 36 2e 30 31 31 29 28 rt=01)(x-hp-ip=192.168.246.011)(
hp-probe[1859]: debug: 0060: 78 2d 68 70 2d 68 6e 3d 48 50 39 41 35 43 43 43 29 28 78 2d 68 70 2d 70 31 3d 4d 46 47 3a 48 50 x-hp-hn=HP9A5CCC)(x-hp-p1=MFG:HP
hp-probe[1859]: debug: 0080: 3b 4d 44 4c 3a 50 68 6f 74 6f 73 6d 61 72 74 20 43 36 32 30 30 20 73 65 72 69 65 73 3b 43 4d 44 ;MDL:Photosmart C6200 series;CMD
hp-probe[1859]: debug: 00a0: 3a 4d 4c 43 2c 50 43 4c 2c 50 4d 4c 2c 44 57 2d 50 43 4c 2c 44 45 53 4b 4a 45 54 2c 44 59 4e 3b :MLC,PCL,PML,DW-PCL,DESKJET,DYN;
hp-probe[1859]: debug: 00c0: 31 32 38 34 2e 34 44 4c 3a 34 64 2c 34 65 2c 31 3b 43 4c 53 3a 50 52 49 4e 54 45 52 3b 44 45 53 1284.4DL:4d,4e,1;CLS:PRINTER;DES
hp-probe[1859]: debug: 00e0: 3a 43 43 39 38 38 3b 53 4e 3a 4d 59 37 43 49 48 35 31 30 50 30 35 30 36 3b 53 3a 30 33 38 30 38 :CC988;SN:MY7CIH510P0506;S:03808
hp-probe[1859]: debug: 0100: 38 43 34 38 34 30 30 31 30 32 31 30 30 37 38 63 62 30 30 30 30 30 34 31 62 38 30 34 32 37 34 36 8C4840010210078cb0000041b8042746
hp-probe[1859]: debug: 0120: 62 38 30 34 34 38 34 37 62 38 30 34 34 65 34 34 62 38 30 34 36 34 34 38 62 38 30 30 35 35 34 35 b8044847b8044e44b8046448b8005545
hp-probe[1859]: debug: 0140: 62 38 30 34 36 34 3b 5a 3a 30 31 30 32 2c 30 35 30 30 30 30 30 39 30 31 36 61 38 35 30 31 33 32 b80464;Z:0102,05000009016a850132
hp-probe[1859]: debug: 0160: 34 35 30 31 33 32 34 35 30 31 33 32 34 35 30 31 33 32 34 31 30 31 33 32 34 35 21 73 54 f0 a0 29 45013245013245013241013245!sT..)
hp-probe[1859]: debug: 0180: 4d 7d 21 73 55 ac ff ff fa 40 a0 39 c7 3f 29 28 78 2d 68 70 2d 67 75 69 64 3d 30 30 31 65 30 62 M}!sU....@.9.?)(x-hp-guid=001e0b
hp-probe[1859]: debug: 01a0: 39 61 35 63 63 63 29 9a5ccc)
Traceback (most recent call last):
  File "/usr/bin/hp-probe", line 162, in <module>
    devices = device.probeDevices([b], timeout, ttl, filter_dict, search, method)
  File "/usr/share/hplip/base/device.py", line 487, in probeDevices
    detected_devices = slp.detectNetworkDevices(ttl, timeout)
  File "/usr/share/hplip/base/slp.py", line 115, in detectNe...

Read more...

Created attachment 1429138
File: backtrace

Created attachment 1429139
File: cgroup

Created attachment 1429140
File: cpuinfo

Created attachment 1429141
File: environ

Created attachment 1429142
File: mountinfo

Created attachment 1429143
File: namespaces

Created attachment 1429144
File: open_fds

Created attachment 1429145
File: hp-probe.log

Created attachment 1528767
Patch provided in Launchpad bug

Patch for /usr/share/hplip/base/sixext.py provided in the Launchpad bug for the same issue.

This is exposed by the switch to using Python3 by default. It's a known bug in HPlip that other distributions have seen when they switched to Python3 by default. It is still present in Fedora 29 and the new HPlip. There's a patch that makes things work (but is probably not the Right Way) by ignoring the UTF-8 conversion errors. Using the patch, everything works for me (and people on other distros.)

Changed in hplip (Fedora):
importance: Unknown → Undecided
status: Unknown → Confirmed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.