Here's the reproducer that shows that dummy0 is being created in the host network namespace. This is an upstream bug (oversight) so far as I can tell from its purpose.

At the outset I should say this is not related to the actual cause of the autopkgtest failure - explanation of that at the end of this comment.

# what the host sees initially
root@t300chi:~# nmcli con
NAME     UUID                                  TYPE      DEVICE 
USB3hub  85f05002-c66a-4b03-b340-21f512ac72fd  ethernet  eth0   
soggy    b9e78bf8-9e6d-4292-9390-d56876fa8300  wifi      wlp2s0 
virbr0   b09a6211-605c-4d0c-bff7-996c5f58299c  bridge    virbr0 

# add the new network namespace
root@t300chi:~# ip netns add test

# run nmcli in the network namespace
root@t300chi:~# ip netns exec test nmcli connection add type dummy con-name dummy0 ifname dummy0 ip4 10.0.0.2 gw4 10.0.0.1
Connection 'dummy0' (2bdfbb93-f51a-48d8-87f3-d2a2b5a72aee) successfully added.

# check if the interface has been added - strange, it lists dummy0 and the host interfaces!!
root@t300chi:~# ip netns exec test nmcli con
NAME     UUID                                  TYPE      DEVICE 
dummy0   2bdfbb93-f51a-48d8-87f3-d2a2b5a72aee  dummy     dummy0 
USB3hub  85f05002-c66a-4b03-b340-21f512ac72fd  ethernet  eth0   
soggy    b9e78bf8-9e6d-4292-9390-d56876fa8300  wifi      wlp2s0 
virbr0   b09a6211-605c-4d0c-bff7-996c5f58299c  bridge    virbr0 

# activate - note this succeeds on my system for a reason explained below
root@t300chi:~# ip netns exec test nmcli con up dummy0                                          
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConn
ection/7)

# lets check what interfaces exist in the network namespace
root@t300chi:~# ip netns exec test ip link show
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

# aha! so dummy0 isn't in the namespace.

# interface is created on the parent network namespace where NetworkManager daemon is
oot@t300chi:~# nmcli con                                                    
NAME     UUID                                  TYPE      DEVICE                                 
dummy0   2bdfbb93-f51a-48d8-87f3-d2a2b5a72aee  dummy     dummy0 
USB3hub  85f05002-c66a-4b03-b340-21f512ac72fd  ethernet  eth0   
soggy    b9e78bf8-9e6d-4292-9390-d56876fa8300  wifi      wlp2s0 
virbr0   b09a6211-605c-4d0c-bff7-996c5f58299c  bridge    virbr0 

---

So the expectation that dummy0 is being created in the network namespace is incorrect. nmcli is talking to NetworkManager over DBus which is a daemon in the parent namespace.

Why is it failing on Ubuntu (but not on my system) ?

Due to my override of 10-globally-managed-devices.conf:

root@t300chi:~# ls -l /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf /etc/NetworkManager/conf.d/

-rw-r--r-- 1 root root   80 Sep 27 16:07 /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf

/etc/NetworkManager/conf.d/:
total 4
-rw-r--r-- 1 root root  0 Oct 31 04:48 10-globally-managed-devices.conf
-rw-r--r-- 1 root root 32 Sep 27 16:07 default-wifi-powersave-on.conf 

root@t300chi:~# cat /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf 
[keyfile]
unmanaged-devices=*,except:type:wifi,except:type:gsm,except:type:cdma

The "unmanaged" result is due to this exclusion that was originally added presumably to only have NM manage wireless devices. I have to disable it to manage Ethernet connections but the wildcard "*" will include type:dummy as unmanaged.