systemd-networkd crashes with SIGSEGV

[Tiago Pasqualini da Silva]

[ Impact ]

Customer reported that this issue happens randomly during their automated regression runs. It's not consistent and hard to reproduce. They have shared a coredump:

Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000555962aa1c34 in link_drop_foreign_config (link=link@entry=0x555963583f30) at ../src/network/networkd-link.c:2741
2741 ../src/network/networkd-link.c: No such file or directory.
(gdb) bt
#0 0x0000555962aa1c34 in link_drop_foreign_config (link=link@entry=0x555963583f30) at ../src/network/networkd-link.c:2741
#1 0x0000555962aa233d in link_carrier_lost.lto_priv.328 (link=<optimized out>, link=<optimized out>) at ../src/network/networkd-link.c:3462
#2 0x0000555962a8e9b2 in link_update (m=0x5559635702c0, link=<optimized out>) at ../src/network/networkd-link.c:3698
#3 manager_rtnl_process_link (rtnl=<optimized out>, message=0x5559635702c0, userdata=<optimized out>) at ../src/network/networkd-manager.c:713
#4 0x0000555962a48a16 in process_match (m=0x5559635702c0, rtnl=0x55596355d990) at ../src/libsystemd/sd-netlink/sd-netlink.c:388
#5 process_running (ret=0x0, rtnl=0x55596355d990) at ../src/libsystemd/sd-netlink/sd-netlink.c:418
#6 sd_netlink_process (rtnl=0x55596355d990, ret=ret@entry=0x0) at ../src/libsystemd/sd-netlink/sd-netlink.c:452
#7 0x0000555962a48cb3 in time_callback (s=<optimized out>, usec=<optimized out>, userdata=<optimized out>) at ../src/libsystemd/sd-netlink/sd-netlink.c:759
#8 0x0000555962a4dbae in source_dispatch (s=s@entry=0x55596355dce0) at ../src/libsystemd/sd-event/sd-event.c:2311
#9 0x0000555962a4de2a in sd_event_dispatch (e=<optimized out>, e@entry=0x55596355bf70) at ../src/libsystemd/sd-event/sd-event.c:2663
#10 0x0000555962a4dfb9 in sd_event_run (e=<optimized out>, e@entry=0x55596355bf70, timeout=timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:2723
#11 0x0000555962a4e1fb in sd_event_loop (e=<optimized out>) at ../src/libsystemd/sd-event/sd-event.c:2744
#12 0x0000555962a223d6 in main (argc=<optimized out>, argv=<optimized out>) at ../src/network/networkd.c:158

This can be fixed by simply null checking link->network before doing anything with it. Upstream has fixed it on this commit: https://github.com/systemd/systemd/commit/b1b0b42e48303134731e017a108c6c334ef5f4c8

[ Test Plan ]

Since this issue is hard to reproduce, I was able trigger it on gdb, by adding a breakpoint once the manager is setup, getting the loopback device and running link_drop_foreign_config on the loopback device:

(gdb) b src/network/networkd.c:150
Breakpoint 1 at 0x2f38f: file ../src/network/networkd.c, line 150.
(gdb) r
Starting program: /lib/systemd/systemd-networkd
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
eth0: IPv6 successfully enabled
eth0: Gained IPv6LL

Breakpoint 1, main (argc=<optimized out>, argv=<optimized out>) at ../src/network/networkd.c:152
152 log_info("Enumeration completed");
(gdb) p *m->links
$1 = {b = {hash_ops = 0x55555578f230 <trivial_hash_ops>, {indirect = {storage = 0x2e, hash_key = "\200\376yUUU\000\000\001\000\000\000\000\000\000",
        n_entries = 1434074992, n_buckets = 21845, idx_lowest_entry = 256, _pad = "\000\000"}, direct = {
        storage = ".\000\000\000\000\000\000\000\200\376yUUU\000\000\001\000\000\000\000\000\000\000p?zUUU\000\000\000\001\000\000\000\000"}},
    type = HASHMAP_TYPE_PLAIN, has_indirect = false, n_direct_entries = 2, from_pool = true}}
(gdb) set $mylink = (Link **) malloc(sizeof(Link *))
(gdb) call link_get(m, 1, $mylink)
$2 = 0
(gdb) p *$mylink
$3 = (Link *) 0x5555557a3f70
(gdb) p **$mylink
$4 = {manager = 0x555555796940, n_ref = 2, ifindex = 1, master_ifindex = 0, ifname = 0x5555557a0060 "lo", kind = 0x0, iftype = 772,
  state_file = 0x55555579c670 "/run/systemd/netif/links/1", mac = {ether_addr_octet = "\000\000\000\000\000"}, ipv6ll_address = {__in6_u = {
      __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, mtu = 65536, udev_device = 0x0,
  flags = 65609, kernel_operstate = 0 '\000', network = 0x0, state = LINK_STATE_UNMANAGED, operstate = LINK_OPERSTATE_CARRIER, address_messages = 0,
  address_label_messages = 0, route_messages = 0, routing_policy_rule_messages = 0, routing_policy_rule_remove_messages = 0, enslaving = 0,
  addresses = 0x0, addresses_foreign = 0x555555796120, routes = 0x0, routes_foreign = 0x555555796180, addresses_configured = false,
  addresses_ready = false, dhcp_client = 0x0, dhcp_lease = 0x0, lease_file = 0x55555579c620 "/run/systemd/netif/leases/1", original_mtu = 0,
  dhcp4_messages = 0, dhcp4_configured = false, dhcp6_configured = false, ndisc_messages = 0, ndisc_configured = false, ipv4ll = 0x0,
  ipv4ll_address = false, ipv4ll_route = false, static_routes_configured = false, routing_policy_rules_configured = false, setting_mtu = false,
  setting_genmode = false, ipv6_mtu_set = false, pool_addresses = 0x0, dhcp_server = 0x0, ndisc = 0x0, ndisc_rdnss = 0x0, ndisc_dnssl = 0x0, radv = 0x0,
  dhcp6_client = 0x0, rtnl_extended_attrs = true, lldp = 0x0, lldp_file = 0x55555579c6d0 "/run/systemd/netif/lldp/1", lldp_tx_fast = 0,
  lldp_emit_event_source = 0x0, bound_by_links = 0x0, bound_to_links = 0x0}
(gdb) call link_drop_foreign_config(*$mylink)

Program received signal SIGSEGV, Segmentation fault.
0x00005555554aec34 in link_drop_foreign_config (link=0x5555557a3f70) at ../src/network/networkd-link.c:2741
warning: Source file is more recent than executable.
2741 } else if (FLAGS_SET(link->network->keep_configuration, KEEP_CONFIGURATION_STATIC))
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on".
Evaluation of the expression containing the function
(link_drop_foreign_config) will be abandoned.
When the function is done executing, GDB will silently stop.

After applying the patch, the issue does not happen:

(gdb) call link_drop_foreign_config(*$mylink)
$10 = 0

[ Where problems could occur ]

This fix implements a simple null check on a pointer before accessing its contents, so it does not impose any risks of regressions.

[Heitor Alves de Siqueira (halves)]

Thank you for the bug report, Thiago! I did a little bit of housekeeping so that this LP accurately reflects this targeting Ubuntu Pro 18.04.

Could you please fill in the description according to the SRU template?

https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template

[Heitor Alves de Siqueira (halves)]

This bug was fixed in the package below, available under Ubuntu Pro 18.04.

systemd (237-3ubuntu10.57+esm1) bionic; urgency=medium

  * d/p/lp2024864-add-missing-null-check.patch:
    - Add NULL check on link_drop_foreign_request (LP: #2024864)

 -- Tiago Pasqualini <email address hidden> Fri, 23 Jun 2023 16:51:01 -0300