Incomplete network info showing in both web and API :: landscape-server 23.03

[Mark Cunningham]

Thank you @mdscunningham for opening the bug report.

[Impact]

Current versions of landscape client omit some network interfaces in their API response. While some interfaces should not be included such as loops and tap interfaces, others such as disconnected interfaces (no IP address or in the down state) should still be included as long as they have a valid MAC address.
This patch makes sure that interfaces are not omitted from the API response based on their IP address or their current state.

[Test Plan]

1- launch a ubuntu instance (lxc container, VM, bare metal) and make sure that the machine has access to at least two network interfaces.
2- Make sure one of the interfaces is connected and has a valid IP.
3- Also make sure at least one interface is not connected and has a valid MAC address. The interface can be in the down state or simply not configured with an IP. One easy way to achieve this is by running the following command:

$ ip link set dev <interface> down

4- Install landscape-client on the machine and enroll it into the landscape server.
5- Wait a few minutes for the machine to enroll and for all information to be exchanged between the server and the client
6- Query the server with lanscape-api to fetch information about the currently connected clients:

 $ landscape-api get-computers --with-network --json

7- Prior to this patch, the interfaces in the down state or with no IP addresses will not show up in the API response. However after this patch these interfaces will show up with an IP address, broadcast address and netmask of 0.0.0.0.

[Where problems could occur]

* Some interfaces might still fail to show up.
* Some interfaces that are not meant to show up might show up.

[Original Description]

Customer has opened a case (00378390) indicating that when they use the lansdcape-api to get network info for client machines they do not get all interfaces returned by the API.

For their specific use case they need to get all MAC addresses from the machine to add those to their VPN for access. They indicate that their wifi interface shows up in the web UI, but not in the API output.

I was able to replicate this in 23.03+18 with my own laptop. Here are my results.

Actual interfaces on machine, there are four, which should all be visible in Landscape.

ip a | grep -E 'BROADCAST|ether|inet '

2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 38:f3:ab:41:dd:0d brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.176/24 brd 192.168.1.255 scope global dynamic noprefixroute enp0s31f6

3: enx803f5d0918d6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 80:3f:5d:09:18:d6 brd ff:ff:ff:ff:ff:ff

4: wlp0s20f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e0:2b:e9:bb:0f:6c brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.143/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp0s20f3

5: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:e3:af:9a brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0

From Web UI, I see three of four:

Network:
Model: Not available
MAC: 80:3f:5d:09:18:d6
IP: Not available

Model: Comet Lake PCH CNVi WiFi
MAC: e0:2b:e9:bb:0f:6c
IP: 192.168.1.143

Model: Ethernet Connection (11) I219-LM
MAC: 38:f3:ab:41:dd:0d
IP: 192.168.1.176

From API, I see a different three of four:

landscape-api get-computers --with-network --query <hostname>

  'network_devices': [{'broadcast_address': '192.168.1.255',
                       'duplex': True,
                       'interface': 'enp0s31f6',
                       'ip_address': '192.168.1.176',
                       'mac_address': '38:f3:ab:41:dd:0d',
                       'netmask': '255.255.255.0',
                       'speed': 1000},
                      {'broadcast_address': '192.168.122.255',
                       'duplex': False,
                       'interface': 'virbr0',
                       'ip_address': '192.168.122.1',
                       'mac_address': '52:54:00:e3:af:9a',
                       'netmask': '255.255.255.0',
                       'speed': 10000},
                      {'broadcast_address': '192.168.1.255',
                       'duplex': False,
                       'interface': 'wlp0s20f3',
                       'ip_address': '192.168.1.143',
                       'mac_address': 'e0:2b:e9:bb:0f:6c',
                       'netmask': '255.255.255.0',
                       'speed': -1}],

I would expect all four interfaces to be visible, and clearly Landscape is aware of all four because if we combine the outputs above you can see all four, but each view Web / API only returns a different three of the four.

[Ghadi Rahme (ghadi-rahme)]

Hi Mark, I opened an upstream PR. It can be tracked here: https://github.com/canonical/landscape-client/pull/241

[Spencer Runde (spencerrunde)]

Ghadi's PR has been merged into main: https://github.com/canonical/landscape-client/pull/241

[Ghadi Rahme (ghadi-rahme)]

Hello,

Attached is the oracular debdiff, I will post the other debdiffs once oracular is accepted.

[Simon Quigley (tsimonq2)]

Sponsored, thanks!

In the future, please use DEP-3 headers: https://dep-team.pages.debian.net/deps/dep3/

[Ghadi Rahme (ghadi-rahme)]

Hello @tsimonq2,

Thank you sponsoring the patch. I wanted to clarify the build failures that are occurring for oracular, these failures are not related to my patch and are a separate issue affecting oracular. The issue is being tracked in a separate LP bug: https://bugs.launchpad.net/landscape-client/+bug/2068050

[Launchpad Janitor (janitor)]

This bug was fixed in the package landscape-client - 24.02-0ubuntu8

---------------
landscape-client (24.02-0ubuntu8) oracular; urgency=medium

  * d/p/2068050-fix-pycurl-error-string-test: Fix difference in pycurl error
    string upon certificate validation failured (LP: #2068050)

 -- Mitch Burton <email address hidden> Wed, 12 Jun 2024 14:51:00 -0700

[Ghadi Rahme (ghadi-rahme)]

Hello @tsimonq2,

It seems that there has been some confusion with the SRU. My patch got labeled as fix released although I do not see it in the changelog.
Also the patch from https://bugs.launchpad.net/landscape-client/+bug/2068050 somehow marked mine as fixed released by Launchpad Janitor.

[Brian Murray (brian-murray)]

Ubuntu 23.10 (Mantic Minotaur) has reached end of life, so this bug will not be fixed for that specific release.

[Ghadi Rahme (ghadi-rahme)]

Attached is the Noble patch

[Ghadi Rahme (ghadi-rahme)]

Attached is the jammy patch

[Ghadi Rahme (ghadi-rahme)]

Attached is the focal patch

[Ghadi Rahme (ghadi-rahme)]

Hello,

Above are the patches for the different Ubuntu releases. Please note that there has been an issue in the oracular patch during the SRU process where my changelog is not showing up, however the patch has been applied.It seems like my changelog was overwritten by another changelog that fixes a bug reported in another LP bug page.
I believe such an issue should be resolved so that users can track what changes are being made to the package.

Also I did not quiet understand how my patch was not dep-3 conformant. If it is still the case with the ones I just uploaded please let me know what is missing/needs change.

Thank you in advance!

Regards,
Ghadi

[Dariusz Gadomski (dgadomski)]

The change is already present in oracular.

[Andreas Hasenack (ahasenack)]

> 7- Prior to this patch, the interfaces in the down state or with no IP addresses will not show up in the
> API response. However after this patch these interfaces will show up with an IP address, broadcast
> address and netmask of 0.0.0.0.

I have some questions about this change.

a) Won't this potentially pollute and overcrowd the landscape server monitoring page? I vaguely remember cases where landscape-client was installed on a system with multiple network interfaces and vlans (I think it was an openstack neutron gateway), and the monitoring page would just be HUGE, with empty graphs because some interfaces didn't have an IP. [citation needed] Or is this change only affecting the API call and its results?

b) Wouldn't it be better to, instead of faking the IP layer with 0.0.0.0, to have an indication that the interface is down, and have that available in the API as a filter perhaps? This would have to be done in a backwards compatible way, of course.

[Ghadi Rahme (ghadi-rahme)]

Hi Andreas,

Thank you for looking into this!

To answer your questions/concerns:

a) These changes only apply for the API and will not result in any change for the UI. I actually also had to confirm that upstream (https://github.com/canonical/landscape-client/pull/241#issuecomment-2139980751).
Also I think such an issue with the UI should be reported and fixed in a different LP bug report (in case there isn't already one) since it would be already affecting users with multiple active interfaces.

b) I had though about this as well while working on this patch and I am still open to modifying it upstream and re-doing the SRU. The initial reason if I remember correctly on why I decided not to, was to avoid changing the output format of the API since it might break some user tools that were developed with this order in mind (blindly iterating over each entry in the json without checking the name).
As for why I used 0.0.0.0 specifically is because of this :"A host may use 0.0.0.0 as its own source address in IP when it has not yet been assigned an address, such as when sending the initial DHCPDISCOVER packet when using DHCP."[1].

Let me know what you think and how I should proceed.

Thank you again for reviewing this patch!

[1] https://en.wikipedia.org/wiki/0.0.0.0

[Andreas Hasenack (ahasenack)]

> a) These changes only apply for the API and will not result in any change for the UI

Great, one less concern

> b) I had though about this as well

I understand 0.0.0.0 is "correct" for such interfaces, thanks. What I think is the surprising factor here is that if someone was using the API to, say, generate their own monitoring graphs, because they didn't like the landscape ones, they will be impacted by this change. Suddenly their api results will potentially have many more interfaces, and they will have to take action to filter them. Or not. That's why I thought maybe a different api call could be used, or perhaps preferably, add an optional argument like "--all", to really list all interfaces. That would not change existing behavior, and whoever wants to see the missing ones can opt in.

What do you think?

[Mauricio Faria de Oliveira (mfo)]

Status: the SRUs are blocked on a reply to Andreas's comment #16, item b).

[Ghadi Rahme (ghadi-rahme)]

Hello this is a summary of what has been decided for the new implementation of this change.

The current changes in landscape-client will remain the same, however to avoid changing the output of the API, a new flag will be defined when doing an API request that would tell landscape server to post these extra interfaces.
Some concerns were discussed and confirmed to be a non issue by the landscape team, mainly the extra load on the server for having to process many more interfaces especially in cloud environment was deemed to be a non issue for current deployments.

The currently proposed landscape client SRU should be put on hold and only merged once the changes to landscape server and landscape-api are posted.

Hope this clarify things!

[Dan Bungert (dbungert)]

Untagging for sponsors, please resubscribe sponsors when the next step is ready for review. Thanks!

[Steve Langasek (vorlon)]

My understanding of the previous comment is that this should not be released to customers until landscape server and api are updated, but that it will not require a reupload of landscape-client vs what is currently in the queue. I am therefore accepting these SRUs and setting the 'block-proposed' tag so that they stay in -proposed until the above is resolved; once it is resolved, please remove the bug tag.

If my understanding is incorrect and the landscape-client package WILL require changes after the server/api changes land, then please tag this bug verification-failed-noble instead and we will remove it again from -proposed.

[Steve Langasek (vorlon)]

Hello Mark, or anyone else affected,

Accepted landscape-client into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/24.02-0ubuntu5.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Mark, or anyone else affected,

Accepted landscape-client into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/23.02-0ubuntu1~22.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Mark, or anyone else affected,

Accepted landscape-client into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/23.02-0ubuntu1~20.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ghadi Rahme (ghadi-rahme)]

Hello,

The changes for landscape-server were merged upstream and added a new API method --with-all-networks to the server as well as landscape-api. This new method will enable the user to display all the network interfaces advertised by landscape-client as discussed internally with @ahasenack. In the meantime this data will be included as part of the --with-network method which will automatically revert to its original behavior when a new version of landscape-server releases with the upstream patch.

Since the version of landscape-server is not out yet all verification will be done with --with-network instead, but keep in mind that this is a temporary behavior for this method until --with-all-networks is made available through the server.

[Ghadi Rahme (ghadi-rahme)]

Focal verification:

1- install landscape client on a machine that is running focal, the proposed version of landscape-client and has at least one interface in the DOWN state (or with no IP address).

root@build-focal:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
16: eth0@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:b6:2d:33 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.30.157.254/24 metric 100 brd 10.30.157.255 scope global dynamic eth0
       valid_lft 3334sec preferred_lft 3334sec
    inet6 fe80::216:3eff:feb6:2d33/64 scope link
       valid_lft forever preferred_lft forever
18: eth2@if19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:8c:f6:f1 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::216:3eff:fe8c:f6f1/64 scope link
       valid_lft forever preferred_lft forever
20: eth6@if21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:f9:03:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.126.242/24 metric 100 brd 192.168.126.255 scope global dynamic eth6
       valid_lft 3334sec preferred_lft 3334sec
    inet6 fe80::216:3eff:fef9:3b7/64 scope link
       valid_lft forever preferred_lft forever

root@build-focal:~# cat /etc/os-release | grep VERSION
VERSION="20.04.6 LTS (Focal Fossa)"
VERSION_ID="20.04"
VERSION_CODENAME=focal

root@build-focal:~# apt list landscape-client
Listing... Done
landscape-client/focal-proposed,now 23.02-0ubuntu1~20.04.4 amd64 [installed]

In this case eth2 is an interface with no IP address, but a valid MAC address.

2- Running landscape-api get-computers --with-network --json we get the following output section for the network devices of the machine I am testing on (full output truncated for readability):
...
"network_devices": [
      {
        "interface": "eth0",
        "ip_address": "10.30.157.254",
        "mac_address": "00:16:3e:b6:2d:33",
        "broadcast_address": "10.30.157.255",
        "netmask": "255.255.255.0",
        "speed": 10000,
        "duplex": true
      },
      {
        "interface": "eth2",
        "ip_address": "0.0.0.0",
        "mac_address": "00:16:3e:8c:f6:f1",
        "broadcast_address": "0.0.0.0",
        "netmask": "0.0.0.0",
        "speed": 10000,
        "duplex": true
      },
      {
        "interface": "eth6",
        "ip_address": "192.168.126.242",
        "mac_address": "00:16:3e:f9:03:b7",
        "broadcast_address": "192.168.126.255",
        "netmask": "255.255.255.0",
        "speed": 10000,
        "duplex": true
      }
    ]
...

As we can see all three interface show up including eth2 with the correct placeholders used for an interface with no IP address.

[Ghadi Rahme (ghadi-rahme)]

jammy verification:

1- install landscape client on a machine that is running jammy, the proposed version of landscape-client and has at least one interface in the DOWN state (or with no IP address).

root@jammy-lds:~# apt list landscape-client
Listing... Done
landscape-client/jammy-proposed,now 23.02-0ubuntu1~22.04.4 amd64 [installed]
N: There are 2 additional versions. Please use the '-a' switch to see them.

root@jammy-lds:~# cat /etc/os-release | grep VERSION
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy

root@jammy-lds:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
36: eth0@if37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:7a:c5:b2 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.30.157.143/24 metric 100 brd 10.30.157.255 scope global dynamic eth0
       valid_lft 2395sec preferred_lft 2395sec
    inet6 fe80::216:3eff:fe7a:c5b2/64 scope link
       valid_lft forever preferred_lft forever
38: eth1@if39: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:f1:ad:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.126.246/24 metric 100 brd 192.168.126.255 scope global dynamic eth1
       valid_lft 2274sec preferred_lft 2274sec
    inet6 fe80::216:3eff:fef1:adc5/64 scope link
       valid_lft forever preferred_lft forever
40: eth2@if41: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:20:78:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0

In this case eth2 is the interface with no IP address

2- Running landscape-api get-computers --with-network --json we get the following output section for the network devices of the machine I am testing on (full output truncated for readability):

...
"network_devices": [
      {
        "interface": "eth0",
        "ip_address": "10.30.157.143",
        "mac_address": "00:16:3e:7a:c5:b2",
        "broadcast_address": "10.30.157.255",
        "netmask": "255.255.255.0",
        "speed": 10000,
        "duplex": true
      },
      {
        "interface": "eth1",
        "ip_address": "192.168.126.246",
        "mac_address": "00:16:3e:f1:ad:c5",
        "broadcast_address": "192.168.126.255",
        "netmask": "255.255.255.0",
        "speed": 10000,
        "duplex": true
      },
      {
        "interface": "eth2",
        "ip_address": "0.0.0.0",
        "mac_address": "00:16:3e:20:78:4d",
        "broadcast_address": "0.0.0.0",
        "netmask": "0.0.0.0",
        "speed": 10000,
        "duplex": true
      }
    ]

...

As we can see all three interface show up including eth2 with the correct placeholders used for an interface with no IP address.

[Ghadi Rahme (ghadi-rahme)]

Noble verification:

1- install landscape client on a machine that is running Noble, the proposed version of landscape-client and has at least one interface in the DOWN state (or with no IP address).

root@build-noble:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
30: eth0@if31: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:49:cc:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.30.157.234/24 metric 100 brd 10.30.157.255 scope global dynamic eth0
       valid_lft 3262sec preferred_lft 3262sec
    inet6 fe80::216:3eff:fe49:cc65/64 scope link
       valid_lft forever preferred_lft forever
32: eth2@if33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:6a:d5:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::216:3eff:fe6a:d570/64 scope link
       valid_lft forever preferred_lft forever
34: eth6@if35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:24:90:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.126.134/24 metric 100 brd 192.168.126.255 scope global dynamic eth6
       valid_lft 3265sec preferred_lft 3265sec
    inet6 fe80::216:3eff:fe24:9015/64 scope link
       valid_lft forever preferred_lft forever

root@build-noble:~# apt list landscape-client
Listing... Done
landscape-client/noble-proposed,now 24.02-0ubuntu5.2 amd64 [installed]
N: There are 2 additional versions. Please use the '-a' switch to see them.

root@build-noble:~# cat /etc/os-release | grep VERSION
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble

In this case eth2 is an interface with no IP address, but a valid MAC address.

2- Running landscape-api get-computers --with-network --json we get the following output section for the network devices of the machine I am testing on (full output truncated for readability):
...

    "network_devices": [
      {
        "interface": "eth0",
        "ip_address": "10.30.157.234",
        "mac_address": "00:16:3e:49:cc:65",
        "broadcast_address": "10.30.157.255",
        "netmask": "255.255.255.0",
        "speed": 10000,
        "duplex": true
      },
      {
        "interface": "eth2",
        "ip_address": "0.0.0.0",
        "mac_address": "00:16:3e:6a:d5:70",
        "broadcast_address": "0.0.0.0",
        "netmask": "0.0.0.0",
        "speed": 10000,
        "duplex": true
      },
      {
        "interface": "eth6",
        "ip_address": "192.168.126.134",
        "mac_address": "00:16:3e:24:90:15",
        "broadcast_address": "192.168.126.255",
        "netmask": "255.255.255.0",
        "speed": 10000,
        "duplex": true
      }
    ]
...

As we can see all three interface show up including eth2 with the correct placeholders used for an interface with no IP address.

[Chris Halse Rogers (raof)]

Hm. In https://bugs.launchpad.net/landscape/+bug/2052834/comments/20 Steve said that he was accepting this but adding block-proposed tags, but I don't see any block-proposed tags being added or removed in the activity log.

In https://bugs.launchpad.net/landscape/+bug/2052834/comments/25 it seems the changes needed to landscape-server have been merged upstream, but not yet released?

So it's not clear to me if this is in a releasable state or not. Are all the pieces necessary to make this fully work released (or releasable) now?

[Ghadi Rahme (ghadi-rahme)]

Hello @raof,

Yes it seems this must have been an oversight from Steve not to add the blocked-proposed tag.

Ideally the patch should be released when the changes in landscape server are also out since this will guarantee no change in the current behavior of the --with-network API command.

If the landscape-client patch is released now, this will cause the --with-network parameter to print all the network data compared to only the active connections.

The new landscape-server patch which is not out yet, will make sure that --with-network only displays the active interfaces as it currently does and a new parameter will allow all of the interfaces to be displayed.

TLDR: Releasing the patch now will cause a change of behavior in the API while waiting for the landscape server release first will avoid this abrupt change and handle things more elegantly.

[Chris Halse Rogers (raof)]

Added the tags so that I don't try to process this again next SRU shift :)

[Timo Aaltonen (tjaalton)]

Any idea when the server will be fixed and this one unblocked? There's https://launchpad.net/bugs/2084586 queued for noble.

[Mitch Burton (mitchburton)]

I can confirm in response to https://bugs.launchpad.net/landscape/+bug/2052834/comments/25 that the server-side changes were released in a beta version of landscape-server (25.04~beta.1) but haven't been released to any stable version.

The Landscape team will make a decision regarding which version of landscape-server to release these server-side changes to this week, which will inform the timing.

I'd like to get other queued changes unblocked - is there a route forward for them that is independent of the timing of the landscape-server releases?

[Mitch Burton (mitchburton)]

We've discussed it within the Landscape team and decided that we will consider the release of the landscape-server-side changes in 25.04~beta.1 sufficient for this to be considered unblocked.

There's no need to re-do verification, as the server-side changes solely affect the user-facing server API and have no impact on landcape-client behaviour.

[Ghadi Rahme (ghadi-rahme)]

Thank you @mitchburton!

Would it be possible to have someone from the SRU team give an ETA on when the release would happen for the client?

[Andreas Hasenack (ahasenack)]

Let's decompose these changes a little bit.

1) message exchanges between landscape-client and server
This is what this SRU is changing. It is adding network information for interfaces that have no IP.

The patch description is misleading: "* Add non-filtered interfaces to the API response (LP: #2052834)"

This patch is not changing the API. It's just sending to the server information that wasn't sent before.

If that information will be returned via an API call, existing or new, that's something between the server and the API client to negotiate/change/decide.

What must NOT happen is the extra information that this new client is now including in its exchanges with the server, break said exchanges. I.e., an old server, when processing these messages from the new (this SRU) client, must NOT break, and either just accept the new data and store it (preferably), or ignore the extra data.

2) changes to API
The API modification is a change in the server and in the landscape-api packages, both of which are not in the ubuntu archive, and was discussed before. Comment #33 states that landscape-server 25.04~beta1 has the API changes, and these do not impact src:landscape-client, subject of this SRU.

In terms of possible regressions, the concerns were/are:

a) extra traffic between all clients and a potentially single server (think landscape.canonical.com, or any dedicated server installed on-prem). This was responded to in comment #18 and deemed irrelevant.

b) Old landscape servers must not break processing of messages by the new client (this SRU). If that happens, the client will become "wedged", and the server will not be able to manage it anymore. This is a deal breaker.

c) Other changes in behavior. The landscape-api package, and its main binary/script, are not part of src:landscape-client. That being said, the plan being put in motion by this SRU will reach the API of course (that's the whole point). But since both landscape-server and landscape-api are NOT in the ubuntu archive, they are not subject to SRU rules. I would caution you, though, to think carefully about changes in behavior regardless.

c1) For example, if I have a script that uses the API to poll the server for network information for certain computers, I would expect the structure of the API response to not change between upgrades. I.e., let's say I'm not aware of the new API parameter, so I don't use it. Then I wouldn't expect the response to include more data than before. This is debatable: if the structure of the response it the same, but just has more fields, then maybe my script shouldn't fail and just ignore the extra data.

c2) If I update the api client and see that the call I have been using has a new parameter, and then I use this new parameter, I would expect the response to change, and that's ok. Furthermore, I would NOT expect this new parameter to break the API call if I contact an old landscape server.

For this SRU, my main concerns were (a) and (b). I believe (a) was addressed. Can someone comment on (b) please? And feel free to comment on the (c) points as well, maybe I got something wrong.

[Ghadi Rahme (ghadi-rahme)]

Hello Andreas. Thank you for looking into this SRU!

To answer your questions/concerns:
a) You are correct, this has already been mentioned by the landscape team as a none concern with the current architecture.

b) The vast majority of my testing was actually done using the old landscape server with 0 changes done to it! The server and the API will continue to work as expected, with the only different being that on old landscape-server deployments the extra network information will be part of the "--with-networks" API call instead of being only included in the new API call "--with-all-networks" implemented in the new landscape-server version.
The changes done to landscape-server were done very late on after after getting feedback that it would be better to separate the behavior into a different API call.
The patch itself was developed and tested using the old landscape-server.

c1) The data returned by "--with-networks" will NOT change if the user upgrades to the new landscape-server and this was the reason for updating landscape-server.
Having said so, users who remain on the old landscape-server will see that there are extra interfaces that will be returned by the "--with-networks" API call, however the format of the message is the exact same as before (even the order in which the parameters are printed), the only difference would be the values of the extra interfaces: For example, these interfaces will show their IP address, broadcast_address and netmask as 0.0.0.0 instead of a valid value. The MAC address will also be the actual MAC address of the interface since interfaces with no MAC address are ignored.
For a more detailed example, Comment #26 #27 and #28 where I did the verification for focal, jammy and noble respectively shows how the response would look like for an interface that is not active, compared to the other active ones. This is also how the output will look like on the new landscape-server when using the new "--with-all-networks" call.

c2) Using a new API call on an old version of landscape-server will result in an error being returned by the API client, but will not result in a crash of landscape-server.

[Mitch Burton (mitchburton)]

Responses from the Landscape team regarding ahasenack's concerns:

a) Has been addressed as mentioned above in comment #18

b) We've established that old landscape servers do not break processing of messages from clients with these new network devices included. In fact, the processing of these messages was not even changed to accommodate any of the changes in this SRU, and has not changed for many landscape-server versions.

c1) There are no changes in structure of the API response between upgrades - the response is an array of computers, each of which contains an array of network devices. This change increases the number of network devices in the inner array but does not affect their structure or the structure of any other part of the response.

c2) We've established that extra parameters made in API calls to older versions of landscape-server are simply ignored. There's no concern regarding new parameters breaking API calls to old landscape-servers.

I believe this addresses the remaining concerns above.

[Mitch Burton (mitchburton)]

Adding to ghadi-rahme's comment #37 (sorry I didn't see it when I was writing my comment).

c2) The API client does give an error is you use it like

landscape-api get-computers --with-all-networks

and it is a version of the API client that is unaware of the new parameters.

However, you can use it like so

landscape-api call GetComputers with_all_networks=true

And it will work, even against an older landscape-server. An older server will ignore the parameter though.

[Andreas Hasenack (ahasenack)]

Ok, this looks good. This is ready to release to updates as soon as the noble 24.04.2 point release is done, in my opinion.

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for landscape-client has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package landscape-client - 24.02-0ubuntu5.2

---------------
landscape-client (24.02-0ubuntu5.2) noble; urgency=medium

  * d/p/add-non-filtered-interfaces-to-the-api-response.patch: include
    all non-filtered network interfaces that have a valid MAC address
    in the API response. (LP: #2052834)

 -- Ghadi Elie Rahme <email address hidden> Mon, 29 Jul 2024 11:31:56 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package landscape-client - 23.02-0ubuntu1~22.04.4

---------------
landscape-client (23.02-0ubuntu1~22.04.4) jammy; urgency=medium

  * d/p/add-non-filtered-interfaces-to-the-api-response.patch: include
    all non-filtered network interfaces that have a valid MAC address
    in the API response. (LP: #2052834)

 -- Ghadi Elie Rahme <email address hidden> Mon, 29 Jul 2024 12:25:04 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package landscape-client - 23.02-0ubuntu1~20.04.4

---------------
landscape-client (23.02-0ubuntu1~20.04.4) focal; urgency=medium

  * d/p/add-non-filtered-interfaces-to-the-api-response.patch: include
    all non-filtered network interfaces that have a valid MAC address
    in the API response. (LP: #2052834)

 -- Ghadi Elie Rahme <email address hidden> Mon, 29 Jul 2024 13:20:01 +0000