A system with rootfs on iSCSI stops booting when ufw.service starts.
The kernel logs iSCSI command/reset timeout until I/O fails and the
root filesystem/journal break.
The issue is that ufw_start() sets the default policy _first_, then
adds rules _later_.
So, a default INPUT policy of DROP (default setting in ufw) prevents
further access to the root filesystem (blocks incoming iSCSI traffic)
thus any rules that could help are not loaded (nor anything else.)
[Fix]
The fix is to set default policy after loading rules in ufw_start().
That seems to be OK as `ip[6]tables-restore -n/--noflush` is used,
and per iptables source, that only sets the chain policy.
The comparison of `iptables -L` before/after shows no differences
(verified on a local rootfs); `run_tests.sh` has 0 skipped/errors.
[Impact]
A system with rootfs on iSCSI stops booting when ufw.service starts.
The kernel logs iSCSI command/reset timeout until I/O fails and the
root filesystem/journal break.
The issue is that ufw_start() sets the default policy _first_, then
adds rules _later_.
So, a default INPUT policy of DROP (default setting in ufw) prevents
further access to the root filesystem (blocks incoming iSCSI traffic)
thus any rules that could help are not loaded (nor anything else.)
[Fix]
The fix is to set default policy after loading rules in ufw_start(). restore -n/--noflush` is used,
That seems to be OK as `ip[6]tables-
and per iptables source, that only sets the chain policy.
The comparison of `iptables -L` before/after shows no differences
(verified on a local rootfs); `run_tests.sh` has 0 skipped/errors.
------- ------- ------- --- ------- ------- ---
Functional tests summary
-------
Attempted: 22 (3339 individual tests)
Skipped: 0
Errors: 0
[ufw info]
# ufw --version
ufw 0.36
Copyright 2008-2015 Canonical Ltd.
# lsb_release -cs
focal
[Boot Log]
[ 232.168355] iBFT detected. init-premount ... done. cmd_timed_ out scsi cmd 00000000310a6696 timedout cmd_timed_ out return shutdown or nh cmd_timed_ out scsi cmd 0000000094ad9246 timedout cmd_timed_ out return shutdown or nh task_mgmt_ fn tmf set timeout cmd_timed_ out scsi cmd 000000001a9458b5 timedout cmd_timed_ out return shutdown or nh device_ reset LU Reset [sc 00000000310a6696 lun 1] device_ reset dev reset result = FAILED target_ reset tgt Reset [sc 00000000310a6696 tgt <...>] target_ reset tgt <...> reset result = FAILED session_ reset wait for relogin restor: 2648 blocked for more than 124 seconds. session_ reset failing session reset: Could not log back into <...> [age 0] DID_TRANSPORT_ DISRUPTED driverbyte= DRIVER_ OK cmd_age=169s to_tree: 1004: inode #1398: lblock 0: comm systemd: error -5 reading directory block
Begin: Running /scripts/
Begin: Mounting root file system ... Begin: Running /scripts/local-top ...
Setting up software interface enp45s0f0np0
...
[ 254.644505] Loading iSCSI transport class v2.0-870.
[ 254.714938] iscsi: registered transport (tcp)
[ 254.780129] scsi host12: iSCSI Initiator over TCP/IP
...
[ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB)
...
[ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
...
[ 266.620860] systemd[1]: Starting Uncomplicated firewall...
Starting Uncomplicated firewall...
...
[ 298.491560] session1: iscsi_eh_
[ 298.580803] session1: iscsi_eh_
[ 298.656262] session1: iscsi_eh_
[ 298.745237] session1: iscsi_eh_
[ 298.745270] session1: iscsi_eh_abort aborting sc 00000000310a6696
[ 298.899644] session1: iscsi_eh_abort aborting [sc 00000000310a6696 itt 0x13]
[ 298.985788] session1: iscsi_exec_
[ 302.075554] session1: iscsi_eh_
[ 302.164786] session1: iscsi_eh_
[ 314.107541] session1: iscsi_tmf_timedout tmf timedout
[ 314.169797] connection1:0: detected conn error (1021)
[ 314.232266] session1: iscsi_eh_abort abort failed [sc 00000000310a6696 itt 0x13]
[ 314.323531] session1: iscsi_eh_abort aborting sc 0000000094ad9246
[ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed.
[ 314.495578] session1: iscsi_eh_abort aborting sc 000000001a9458b5
[ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed.
[ 314.664050] session1: iscsi_eh_
[ 314.755773] session1: iscsi_eh_
[ 314.834736] session1: iscsi_eh_
[ 314.954144] session1: iscsi_eh_
[ 315.063456] connection1:0: detected conn error (1021)
[ 315.125743] session1: iscsi_eh_
[ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds.
...
[ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds.
...
[ 402.483917] INFO: task iptables-
...
[ 435.707549] session1: session recovery timed out after 120 secs
[ 435.780058] session1: iscsi_eh_
[ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery
[ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=
[ 436.015520] sd 12:0:0:1: rejecting I/O to offline device
[ 436.134354] sd 12:0:0:1: [sda] tag#105 CDB: Read(10) 28 00 00 05 8d d8 00 00 08 00
[ 436.198807] blk_update_request: I/O error, dev sda, sector 360816 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[ 436.198818] blk_update_request: I/O error, dev sda, sector 2324480 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
[ 436.198852] EXT4-fs warning (device sda1): htree_dirblock_
[ 436.290259] blk_update_request: I/O error, dev sda, sector 363992 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[ 436.417093] Buffer I/O error on dev sda1, logical block 262144, lost sync page write
[ 436.417103] blk_update_request: I/O error, dev sda, sector 364040 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[ 436.417130] JBD2: Error -5 detected when updating journal superblock for sda1-8.
[ 436.417132] Aborting journal on device sda1-8.