ufw errors during boot with upstart (/tmp not available)

[Roman Yepishev]

ufw started emiting errors on boot, the same configuration worked perfectly on Karmic.
I was not able to find any traces of these errors in syslog-related files

The errors are emitted by iptables-restore, though I am not really sure what is being restored

init: ufw pre-start process (985) terminated with status 1
iptables-restore: line 2 failed
iptables-restore: line 2 failed
ip6tables-restore: line 2 failed
ip6tables-restore: line 2 failed
init: ufw pre-start process (1515) terminated with status 1
^ this repeats 3 times then modem-manager starts to print its plugins.

rtg@buzz:~/Videos$ ufw --version
ufw 0.29.3-0ubuntu1
Copyright 2008-2009 Canonical Ltd.

When system is finally booted it looks like it does not have ufw rules in iptables (will doublecheck this on next reboot).
When I attempt to enable ufw later on:

rtg@buzz:~/Videos$ sudo ufw enable
ERROR: problem running ufw-init
rtg@buzz:~/Videos$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: reject (incoming), allow (outgoing)
New profiles: skip

To Action From
-- ------ ----
Anywhere ALLOW IN 192.168.1.0/24
80/tcp (Apache) ALLOW IN Anywhere
22 ALLOW IN Anywhere
5060 ALLOW IN Anywhere
Anywhere ALLOW IN 192.168.122.0/24
24220 ALLOW IN Anywhere
Anywhere (v6) ALLOW IN 2001:470:1f0b:cfb::2/64
80/tcp (Apache (v6)) ALLOW IN Anywhere (v6)
22 ALLOW IN Anywhere (v6)
113/tcp ALLOW IN Anywhere (v6)
5060 ALLOW IN Anywhere (v6)
24220 ALLOW IN Anywhere (v6)

However, this is what gets in INPUT chain (libvirt is installed and being used)
Chain INPUT (policy DROP 93 packets, 25012 bytes)
 pkts bytes target prot opt in out source destination
    0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53
    0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53
    0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67
    0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67

And all legitimate INPUT is being blocked.

I am attaching the content of my /lib/ufw and /etc/ufw directory.

I will be happy to provide any additional information. I will try to reproduce this on vm in case it is not clear what is happening from the description above.

P.S. i have ipv6-enabled link, so v6 rules really have to be there.

[Jamie Strandboge (jdstrand)]

Thank you for using Ubuntu and taking the time to report a bug.

ufw saves its files as iptables-restore style files (/etc/ufw/before*rules, /lib/ufw/user*rules and /etc/ufw/after*rules). Can you run the following commands and attach the resulting file (/tmp/521358.txt):
$ sudo sh -x /lib/ufw/ufw-init stop > /tmp/521358.txt 2>&1
$ sudo sh -x /lib/ufw/ufw-init start >> /tmp/521358.txt 2>&1

[Roman Yepishev (rye)]

Thanks for quick response!

The requested shell script trace is attached.

[Jamie Strandboge (jdstrand)]

Can you reboot, then attach the output of this command:
$ sudo ufw show raw

[Roman Yepishev (rye)]

Hm, the boot procedure ended up with this printed to the terminal and I was able to reproduce with ufw-init start.

root@buzz:/lib/ufw# ./ufw-init start
iptables-restore: line 2 failed
iptables-restore: line 2 failed
iptables-restore v1.4.4: Couldn't load target `ufw-logging-deny':/lib/xtables/libipt_ufw-logging-deny.so: cannot open shared object file: No such file or directory

Error occurred at line: 29
Try `iptables-restore -h' or 'iptables-restore --help' for more information.
iptables-restore v1.4.4: Couldn't load target `ufw-skip-to-policy-input':/lib/xtables/libipt_ufw-skip-to-policy-input.so: cannot open shared object file: No such file or directory

Error occurred at line: 19
Try `iptables-restore -h' or 'iptables-restore --help' for more information.
iptables-restore v1.4.4: Couldn't load target `ufw-user-input':/lib/xtables/libipt_ufw-user-input.so: cannot open shared object file: No such file or directory

Error occurred at line: 2
Try `iptables-restore -h' or 'iptables-restore --help' for more information.
ip6tables-restore: line 2 failed
ip6tables-restore: line 2 failed
ip6tables-restore v1.4.4: Couldn't load target `ufw6-logging-deny':/lib/xtables/libip6t_ufw6-logging-deny.so: cannot open shared object file: No such file or directory

Error occurred at line: 34
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
ip6tables-restore v1.4.4: Couldn't load target `ufw6-skip-to-policy-input':/lib/xtables/libip6t_ufw6-skip-to-policy-input.so: cannot open shared object file: No such file or directory

Error occurred at line: 19
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
ip6tables-restore v1.4.4: Couldn't load target `ufw6-user-input':/lib/xtables/libip6t_ufw6-user-input.so: cannot open shared object file: No such file or directory

Error occurred at line: 2
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.

I went somehow further and wrapped ip[6]tables-restore with a script that captured stdin and printed it out. I am attaching the output.

It looks like ufw-user-input chains are not properly created, though i may be mistaking...

[Roman Yepishev (rye)]

Ok, this starts to be interesting. Out of 3 consecutive reboots the tables were initialized once.

Additionally, if I switch to tty1 here's what is printed there (scrollback buffer is cleared so I can't actually see the beginning):

Error occurred at line: 19
Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
/lib/ufw/ufw-init-functions: line 341: cannot create temp file for here-document: Read-only file system
/lib/ufw/ufw-init-functions: line 376: cannot create temp file for here-document: Read-only file system

Problem running '/etc/ufw/before.rules'
Problem running '/etc/ufw/after.rules'
Problem running '/etc/ufw/before6.rules'
Problem running '/etc/ufw/after6.rules'

I have to admit that I have /boot, /opt and /usr mounted as ro at boot time. But since 'sometimes' it does succeed I think there's probably some racing condition when fs is not ready for this. However I have never heard that here-documents actually required the presence of temp file :-/

Ok, went capturing 'ufw show raw'

[Roman Yepishev (rye)]

Not much different - INPUT table is not initialized properly.

[Roman Yepishev (rye)]

Ok, the here-doc file is created in /tmp (according to bash sources - redir.c and tmpfile.c), which is on / filesystem in Lucid (it was tmpfs in Karmic, hm...).
If we assume that / is still readonly when ufw script runs then it could cause the table initialization to fail.

However, i'd expect much, much more startup errors than I have now.

[Jamie Strandboge (jdstrand)]

Roman, your work was very helpful, thanks!

This is clearly a problem with upstart integration and trying to start ufw very early in the boot process (before any interfaces come online). I saw this in my testing for 0.29.3 which is why I moved to the implementation you see now with here documents. I did not know about the here doc needing a temp file either. I will create a special ufw-init-functions script for you to try that doesn't use /tmp, if it works, I'll roll it out. Otherwise, I'll adjust the upstart job to wait on /tmp (though that is suboptimal...).

[Jamie Strandboge (jdstrand)]

Roman, can you make a backup of /lib/ufw/ufw-init-fucntions and then use this instead?

[Roman Yepishev (rye)]

Thanks!

I have tested with printf-enabled version you've posted - it boots fine and does not fail.
I will test it for one more day and will report here if I run into any issues, though I don't think there will be any.

[Jamie Strandboge (jdstrand)]

ufw (0.29.3-0ubuntu2) lucid; urgency=low

  * fix occasional ufw errors during boot with upstart (/tmp not available)
    (LP: #521359). Cherrypick r633 from trunk.

[elzo valugi (valugi)]

Still having this issue with fresh installs.

[saulius (saulius-m)]

12.04 fresh install + ufw, failed with message about init problems, after that enables well, althow does not start at boot, any ideas?

[Jamie Strandboge (jdstrand)]

elzo and ouzu, I'm sorry for the issues you are seeing, however this bug is closed. Can someone who has exact steps on how to reproduce file a new bug with 'ubuntu-bug ufw'? It might also be useful to adjust /etc/init/ufw.conf to have:
pre-start exec /usr/bin/strace -f -T -v -o /ufw.strace sh -c '/lib/ufw/ufw-init start quiet'

Then reboot and attach /ufw.strace to the new bug.