ufw 0.31: UnicodeDecodeError errors

Bug #953372 reported by Sławomir Nizio on 2012-03-12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Jamie Strandboge
ufw (Ubuntu)
Jamie Strandboge
Jamie Strandboge

Bug Description

I'm sorry to say but apparently change introduced to resolve bug 921758 breaks ufw under some locales, for example sk and es,.
I've noticed it after running a test with various locales today…

I have language files in /usr/share/locale/*/LC_MESSAGES/ufw.mo (that's different from where ufw installs them by default, but it shouldn't make any difference - any location from where .mo are read should should do).

Steps to reproduce.
1. add any rule ($ip - any ip)
ufw deny to $ip
2. 'ufw status' with 'sk' locale fails. Make sure you have the sk translation and it's used.
# LANG=sk ufw status
Traceback (most recent call last):
  File "/usr/sbin/ufw-2.7", line 89, in <module>
    ui = ufw.frontend.UFWFrontend(pr.dryrun)
  File "/usr/lib/python2.7/site-packages/ufw/frontend.py", line 155, in __init__
    self.backend = UFWBackendIptables(dryrun)
  File "/usr/lib/python2.7/site-packages/ufw/backend_iptables.py", line 45, in __init__
    ufw.backend.UFWBackend.__init__(self, "iptables", dryrun, files)
  File "/usr/lib/python2.7/site-packages/ufw/backend.py", line 53, in __init__
  File "/usr/lib/python2.7/site-packages/ufw/backend_iptables.py", line 636, in _read_rules
    tmp[4], tmp[5], dtype)
  File "/usr/lib/python2.7/site-packages/ufw/common.py", line 65, in __init__
  File "/usr/lib/python2.7/site-packages/ufw/common.py", line 186, in set_port
    err_msg = _("Bad port '%s'") % (port)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2: ordinal not in range(128)

Different commands fail in different places. For example when you make this change in common.py, line 186:
- err_msg = _("Bad port '%s'") % (port)
+ err_msg = _("Bad port '%s'") % (port.encode())
(this is only an example, not suggested fix)

that works for this command, but another error occurs:
Traceback (most recent call last):
  File "/usr/sbin/ufw-2.7", line 111, in <module>
    res = ui.do_action(pr.action, "", "", pr.force)
  File "/usr/lib/python2.7/site-packages/ufw/frontend.py", line 578, in do_action
    res = self.get_status()
  File "/usr/lib/python2.7/site-packages/ufw/frontend.py", line 246, in get_status
    out = self.backend.get_status(verbose, show_count)
  File "/usr/lib/python2.7/site-packages/ufw/backend_iptables.py", line 407, in get_status
    full_str += s
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 33: ordinal not in range(128)

Why locale matters - hm, I'm not sure, but perhaps it occurs when it has a translated string (from gettext()) with non-Latin letters and tries to mix it or something with a non-Unicode string -- that could be why it doesn't happen in my tested cases with pl locale - Polish has diacritics, but its ufw translation isn't rich. Please keep in mind I didn't investigate it much, but it looks like this from a shallow view. I'm attaching a file that follows step 2 (after step 1 is done) with different LANG values in case it will be useful.

Related branches

Sławomir Nizio (snizio) wrote :
Changed in ufw:
assignee: nobody → Jamie Strandboge (jdstrand)
importance: Undecided → High
status: New → Confirmed
Changed in ufw (Ubuntu):
assignee: nobody → Jamie Strandboge (jdstrand)
status: New → In Progress
importance: Undecided → High
milestone: none → ubuntu-12.04-beta-2
tags: added: rls-p-tracking
Jamie Strandboge (jdstrand) wrote :

So, even this fails:
sudo LC_ALL=tr_TR.UTF-8 ufw status

So it seems we did not actually fix the original bug correctly. I have test cases now to test this and will revert the fix.

Sławomir Nizio (snizio) wrote :

LC_ALL=tr_TR.UTF-8 ufw status doesn't fail for me. I don't have even Turkish translations (they aren't shipped).

Sławomir Nizio (snizio) wrote :

Hm, anyway, it can fail for you if you have them from other sources than the shipped tarball, but the original issue IS fixed. Just saying. :)

Jamie Strandboge (jdstrand) wrote :

I am actually unable to trigger the "string" vs "strIng" issue at all within ufw itself if I revert the codecs.open() change. Eg, if I apply the following diff to src/backend.py:

=== modified file 'src/backend.py'
--- src/backend.py 2011-05-14 16:03:00 +0000
+++ src/backend.py 2012-03-12 23:03:33 +0000
@@ -205,6 +205,7 @@


+ print self.defaults
         # do some default policy sanity checking
         policies = ['accept', 'accept_no_track', 'drop', 'reject']
         for c in [ 'input', 'output', 'forward' ]:

Then do:
$ sudo LC_TYPE=tr_TR.UTF-8 ufw status
{'ipt_modules': 'nf_conntrack_ftp nf_nat_ftp nf_conntrack_netbios_ns', 'default_output_policy': 'accept', 'loglevel': 'low', 'manage_builtins': 'no', 'enabled': 'no', 'default_input_policy': 'drop', 'default_forward_policy': 'drop', 'ipv6': 'yes', 'default_application_policy': 'skip'}
Status: inactive

You can see that I don't have default_Input_policy, but the proper 'default_Input_policy'. I think that this ufw-frontends needs to be adjusted. Please file a new bug. I am reverting this change in ufw.

Jamie Strandboge (jdstrand) wrote :

I installed the language-pack-tr translations in Ubuntu and cannot trigger the issue in ufw. Please file a new bug with a reduced test case on how ufw itself is failing as opposed to how ufw-frontends is using it. I suggest ufw-frontends explicitly set LC_CTYPE for now.

Jamie Strandboge (jdstrand) wrote :

Meh, comment #5 is confusing. First, the indentation is wrong for the diff, and second I should have said:
"You can see that I don't have default_Input_policy, but the proper 'default_input_policy'."

Jamie Strandboge (jdstrand) wrote :

Sigh, this is the actual test I used to prove to myself that this isn't ufw's fault:
$ sudo LC_ALL=tr_TR.UTF-8 ufw status
{u'ipt_modules': u'nf_conntrack_ftp nf_nat_ftp nf_conntrack_netbios_ns', u'default_output_policy': u'accept', u'loglevel': u'low', u'manage_builtins': u'no', u'enabled': u'no', u'default_input_policy': u'drop', u'default_forward_policy': u'drop', u'ipv6': u'yes', u'default_application_policy': u'skip'}
Durum: kapalı

Changed in ufw:
status: Confirmed → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ufw - 0.31-0ubuntu2

ufw (0.31-0ubuntu2) precise; urgency=low

  * revert r752 which causes ufw to fail with certain locales (LP: #953372)
 -- Jamie Strandboge <email address hidden> Mon, 12 Mar 2012 18:15:05 -0500

Changed in ufw (Ubuntu Precise):
status: In Progress → Fix Released
Sławomir Nizio (snizio) wrote :

The reason why it's all fine with ufw is it doesn't set up locale*. It gets translated strings but doesn't set the environment which is perfectly fine, I think.

How to test that: if you add these lines to /usr/sbin/ufw:

 24 import locale
 25 locale.setlocale(locale.LC_ALL, '')

then you will get this with LC_ALL=tr_TR.UTF-8 ufw status:
{'Ipt_modules': 'nf_conntrack_ftp nf_nat_ftp nf_conntrack_netbios_ns', 'default_applIcatIon_polIcy': 'skIp', 'loglevel': 'low', 'manage_buIltIns': 'no', 'enabled': 'yes', 'default_Input_polIcy': 'drop', 'default_forward_polIcy': 'drop', 'Ipv6': 'yes', 'default_output_polIcy': 'accept'}
ERROR: Missing policy for 'input'

providing that you don't use codecs.open(), but normal open().

What that frontend changes is that it executes ufw code with user's locale and that's why it is OK in ufw but not there.

I guess nothing is to be changed in ufw then. However please keep this in mind in case you want to implement setting locale in ufw at some point.

Thanks for all!

*import locale; locale.getlocale() returns "(None, None)", unlike in that frontend: "('tr_TR', 'UTF-8')" or any

Jamie Strandboge (jdstrand) wrote :

This was fixed in 0.31.1.

Changed in ufw:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Bug attachments