Use ip instead of ifconfig and route

Bug #925145 reported by Garrett Holmstrom
26
This bug affects 4 people
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Medium
Unassigned
Fedora
Confirmed
Medium

Bug Description

ifconfig just changed its output format again, breaking netinfo.py. Since net-tools have been deprecated for some time it makes more sense to switch to iproute as the net-tools maintainers suggest. Attached is a patch for netinfo.py and cc_disable_ec2_metadata.py that does so.

Related branches

Revision history for this message
In , Jiri (jiri-redhat-bugs) wrote :

Hi,

I see that cloud-init uses net-tools (ifconfig and route) for displaying of network interfaces and routes. You maybe know that net-tools have been deprecated for quite a long time and we (as well as other distributions [1][2]) try to eviscerate its use from as many packages as possible.

I've created 2 patches that replace use of ifconfig and route with 'ip addr' and 'ip route'. I could send them to upstream myself but I see there are several patches in spec file so you have probably more experience with this. Will you do that for me ?
If you decide to apply this patches in Fedora then remove the superflous
'Requires: net-tools' line from spec file.

Thanks.

[1] http://lists.debian.org/debian-devel/2009/03/msg00780.html
[2] https://bugzilla.novell.com/show_bug.cgi?id=492665

Revision history for this message
In , Jiri (jiri-redhat-bugs) wrote :

Created attachment 558841
Use 'ip addr' instead of ifconfig

I updated net-tools in rawhide to upstream git snapshot.
The ifconfig output is now little different (see bug #784314).
Fixing cloud-init to understand the new output format is nonsense as
ifconfig has been deprecated for long time and 'ip addr' should be used instead.
And that exactly does my patch.
As you can see the code is now much shorter and more simple
while the output is almost identical
so I think it should be easily acceptable for upstream.

Revision history for this message
In , Jiri (jiri-redhat-bugs) wrote :

Created attachment 558845
Use 'ip route' instead of 'route'

Unlike the previous patch this one could be harder for upstream to accept as the 'ip route' output is quite different from 'route' output. However the main argument for the change remains, i.e. 'route' tool is no longer being developed and 'ip route' plays more nicely with nowadays kernels.

Revision history for this message
Garrett Holmstrom (gholms) wrote :
Revision history for this message
In , Garrett (garrett-redhat-bugs) wrote :

Good idea. I submitted this upstream; let's see what they have to say.

Revision history for this message
Scott Moser (smoser) wrote :

Garrett,
  Thanks for this patch.

Do you have a feeling on how constant the output of 'ip -o route' is?
If we're basically going to pass that through un-modified, I'm hoping its not going to change all the time, so that tools could read it.

The only other comment I have is that I'd join fields on "," not ", " as the output is otherwise only space delimited.
I'll pull this in Monday.

Changed in cloud-init:
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
Jiri Popelka (jpopelka) wrote :

Hi Scott,

my personal opinion is that the 'ip route' and 'ip addr' output won't be ever significantly changed because now most tools in every distribution use them instead of net-tools.
The change in ifconfig output that Garrett mentioned is another case: that happened in upstream repo almost 9 years ago and the reson probably was to make the output of ifconfig similar to the output of ifconfig on Solaris and *BSD.

The patch is just a suggestion. It's of course on you how you implement it.
Please also see the original bug report
https://bugzilla.redhat.com/show_bug.cgi?id=786488

Thanks

Revision history for this message
In , Fedora (fedora-redhat-bugs) wrote :

This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

Revision history for this message
In , Matthew (matthew-redhat-bugs) wrote :

*** Bug 949832 has been marked as a duplicate of this bug. ***

Revision history for this message
Matthew Miller (mattdm) wrote :

Hi Scott! Doesn't look like this got pulled in. It would be Nice To Have. Thanks!

Revision history for this message
Scott Moser (smoser) wrote :

As brought up in bug https://bugs.launchpad.net/cloud-init/+bug/1285185
I think that if we want to do this, it might be best to just use python-netifaces.

Revision history for this message
Scott Moser (smoser) wrote :

note, ipv6 route information is busted, and should be addressed by this also (see bug 1377005).

Revision history for this message
In , Garrett (garrett-redhat-bugs) wrote :
Revision history for this message
In , Garrett (garrett-redhat-bugs) wrote :

*** Bug 1496124 has been marked as a duplicate of this bug. ***

Revision history for this message
In , James (james-redhat-bugs) wrote :

Since the bug opened to cover this, blocking the tracking bug, was closed as a duplicate and that upstream the bug is still open this needs to be re-opened so it actively blocks the tracking bug.

CLOSED/UPSTREAM is valid for the situation it's been fixed upstream and it's just pending a new release upstream to get sucked into Fedora.

As this is an issue that still actively affects Fedora it needs to be open so that it's visible and correctly blocks the tracking bug to keep an eye on the progress.

The assignee has also been reset since this was from 2 years ago and the current package details may have changed.

Revision history for this message
In , James (james-redhat-bugs) wrote :

I did a review of the patch this morning.

Ignoring the BSD etc issues that mean ifconfig probably still needs to be run as a fallback if ip is not found (so unfortunately I don't think it valid to just swap out the util.subp(["ifconfig", "-a"]) in the current code to "ip" instead) ...

It's not just ifconfig and route but also netstat being called that needs to be changed.

I think that should be it though.

Garrett are you in a position to drive this forward or would some assistance be appreciated?

After finishing off facter this looks like the next most troublesome package to fix up.

Changed in fedora:
importance: Unknown → Medium
status: Unknown → Confirmed
Revision history for this message
In , Garrett (garrett-redhat-bugs) wrote :

Since the cloud platforms I have access to don't generally employ the sort of instance-driven network configuration that cloud-init can do I am hesitant to try and fix the issue myself, but I am happy to pull in patches once they exist. If you have the cycles to work on this I would certainly appreciate it. Upstream suggested using python-netifaces as a means of abstracting the platform-specifics away; doing that might make the fallback you mentioned unnecessary.

To be frank, I am *this* close to forking net-tools just so I can go back to using the same network management commands that I use on every other operating system. If all else fails, that would make this issue go away, too.

Revision history for this message
In , James (james-redhat-bugs) wrote :

Upstream both suggested that and declared they wanted to avoid additional dependencies ...

Reading through the launchpad project history and issues ... well ... it's not very well resourced it appears.

I don't have a platform to actively test this on, but now Facter3 is done I do have some availability to update patches and try for a push upstream...

On a side note ... forking net-tools doesn't help the issues at hand as it's fundamental problems with net-tools ... it uses the ioctl API rather than the netlink API which is why it's missing information about routing tables, IP addresses assigned and so on ...

You could re-write net-tools against the netlink API but that would be a lot of effort and would probably get minimal assistance from the network stack kernel maintainers as they already maintain iproute2 themselves ... and would probably be reluctant to maintain two things themselves for the same tasks.

You might want to review this old LWN article: https://lwn.net/Articles/710533/

There's previously been discussion of a wrapper... especially within RHEL where consistency of behaviour is critical ...

https://bugzilla.redhat.com/show_bug.cgi?id=687920#c8

--------------------------------------

This is a comment from the previous net-tools maintainer I found in one (private) RHEL bug, where the wrapper for 'ip' to emulate ifconfig was also discussed:
"we've already tried that but the command line options of both commands
differ significantly. In fedora most of the
scripts (initscripts/networking) has been using iproute commands now and we're
trying to teach customers to switch to ip as well. (iproute has active upstream
following kernel changes closely, immediately bringing features kernel adds
into iproute releases).

Wrapper is one option, but that would have to be properly staffed. We're
talking not only about ifconfig but as well about netstat and other tools
net-tools package is providing. All of them have their equivalents in other
packages, but unfortunately we still have customers relying on net-tools."

---------------------------------------

On the bright side once this work to use iproute2 is complete it is going to be safer looking forwards towards RHEL8 etc.

Revision history for this message
In , gholms (gholms-redhat-bugs) wrote :

Yes, I went down the wrapper route as well some time ago before realizing it would be far more maintainable to port the entire suite to netlink. It isn't like iproute2 is the only thing that uses that interface. As always, it's just a matter of finding enough time to finish it, but if it means Fedora can behave consistently with every other operating system again it will be well worth the effort.

You're right about cloud-init being understaffed. Most of its main maintainers have their hands full with wrangling all the different cloud platforms and operating systems out there that they need to support, so technical debt like this often goes neglected.

Revision history for this message
James Hogarth (james-hogarth) wrote :

A quick update on this ...

My branch has iproute2 (the ip command) used as a preference, but falls back to the net-tools commands if it's not present to maintain compatibility.

I've added tests to cover both net-tools and iproute2 code paths.

I've fixed up the invalid route_info() tests that ignored that the ipv6 route table appears different to the ipv4 one. I've fixed the ipv6 route table generation to actually generate and parse the routes as opposed to printing current socket information. I've added metric to the route table printed as that's useful to determine the actual path.

For the sake of compatibility the net-tools and iproute2 code paths both produce identical output - obviously it'd be nice in future to provide more detail if iproute2 is installed and providing it, but for the sake of a minimal patch to remove the net-tools dependency this seems the best route.

Last thing I need to add is parsing multiple addresses on a single interface.

Once that is done by my count at least the following bugs will be taken care of by the merge: 925145, 1616616 and 1401965 ...

Revision history for this message
James Hogarth (james-hogarth) wrote :
Revision history for this message
Scott Moser (smoser) wrote :

@James,
I hit 'propose for merge'.
It doesn't have to be perfect to do that, we'll review it and give you some fedback.

Thanks for your work.

Revision history for this message
James Hogarth (james-hogarth) wrote :

Sure thing Scott - cheers :)

Since ifconfig changed syntax a while back so RHEL6 and RHEL7 differ, and debian based distros like Ubuntu (and Debian of course) switched recently as well I thought I'd add a test to make sure that it still works ...

Test shows that it doesn't parse correctly and gets the wrong output ... good test! ;)

I'll fix that up first and push that later tonight and then look at the multiple IP addresses issue.

I don't have a BSD box to hand and I'm leaning on too lazy to put a VM together for it given I never to BSD dev work usually ...

Do you have sample ifconfig output from BSD to hand that I can add to the test cases? Is there any difference you are aware of in FreeBSD, NetBSD, etc on ifconfig output?

Is there any formal list of which BSD's you intend to support?

Revision history for this message
James Hogarth (james-hogarth) wrote :

So I note in the codebase there is only a is_freebsd() check so I'm running with the assumption on FreeBSD is in your support matrix?

I built a NetBSD and FreeBSD VMs and the output of ifconfig is different in both.

Given the rest of the codebase I'll just add tests and validate for FreeBSD ... if it turns out you want NetBSD as well that should be fairly trivial to add.

Revision history for this message
In , bcotton (bcotton-redhat-bugs) wrote :

This bug appears to have been reported against 'rawhide' during the Fedora 28 development cycle.
Changing version to '28'.

Dan Watkins (oddbloke)
Changed in cloud-init:
status: Triaged → Fix Released
Revision history for this message
In , fedora-admin-xmlrpc (fedora-admin-xmlrpc-redhat-bugs) wrote :

This package has changed maintainer in Fedora. Reassigning to the new maintainer of this component.

Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.