Make ethtool installed by default in Ubuntu

[bugproxy]

Currently (and since always AFAIK) the ethtool package is not installed by default in Ubuntu on PowerPC.
This is a very important tool for network configuration, troubleshooting and performance report/tunning.

We would like to discuss the possibility to include it by default in the very basic/small Ubuntu installation - it might be really important if for some reason a boot happens to be without network, for example, since by using ethtool user might fix the issue (or at least investigate and collect necessary logs).

Thanks in advance,

Guilherme

[Christian Ehrhardt  (paelzer)]

Hi,
according to [1] and [2] it should be seeded already. Maybe there is an arch dependent twist to the seeding that I fail to see yet.

Also I spawned a new (cloudimg basesd) kvm guest and it had ethtool right away as I expected.
You can see it is seeded since it has no rdepends bringin it in.

ubuntu@z-test:~$ dpkg -l '*ethtool*'
ii ethtool 1:4.8-1 ppc64el

$ apt-cache rdepends --installed ethtool
ethtool
Reverse Depends:

Maybe I'm overlooking something, could you outline on which install from which image and which version exactly that you miss ethtool?

[1]: http://people.canonical.com/~ubuntu-archive/seeds/ubuntu.zesty/server
[2]: http://people.canonical.com/~ubuntu-archive/germinate-output/ubuntu.zesty/server

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-05-12 10:53 EDT-------
(In reply to comment #8)
> Hi,
> according to [1] and [2] it should be seeded already. Maybe there is an arch
> dependent twist to the seeding that I fail to see yet.
>
>
>[...]
>
> Maybe I'm overlooking something, could you outline on which install from
> which image and which version exactly that you miss ethtool?
>
> [1]: http://people.canonical.com/~ubuntu-archive/seeds/ubuntu.zesty/server
> [2]:
> http://people.canonical.com/~ubuntu-archive/germinate-output/ubuntu.zesty/
> server

Thanks paelzer! I tried yesterday the following image, Netboot install:

http://ports.ubuntu.com/ubuntu-ports/dists/zesty/main/installer-ppc64el/current/images/netboot/ubuntu-installer/ppc64el/vmlinux

http://ports.ubuntu.com/ubuntu-ports/dists/zesty/main/installer-ppc64el/current/images/netboot/ubuntu-installer/ppc64el/initrd.gz

I did default installation, without marking additional software to be installed in the end of the installer.
Noticed that ethtool wasn't available after boot.

Cheers,

Guilherme

[Christian Ehrhardt  (paelzer)]

Subscribing nacc/racb so we can discuss the details of seeding in this case on our next sync.

[Robie Basak (racb)]

Looks like ethtool is in the server seed, but not in the standard or minimal seeds. I suspect the netboot install installs only minimal (or perhaps standard).

Perhaps the installer needs to be changed to have the server task selected by default.

[Robie Basak (racb)]

Alternatively it might make sense to put ethtool in the standard seed, if that is what the installer is using by default. I'm not sure it makes sense to put it in the minimal seed though, since it isn't actually necessary for install (only debugging, etc), the minimal seed is supposed to be minimal and we have the standard seed for this use case.

[Dimitri John Ledkov (xnox)]

I thought ethtool is obsolete =/ I don't use it at all

Instead of it shouldn't one be using iproute2? Is there functionality that is missing and e.g. not documented at http://baturin.org/docs/iproute2/ ?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-05-16 10:31 EDT-------
(In reply to comment #12)
> I thought ethtool is obsolete =/ I don't use it at all
>
> Instead of it shouldn't one be using iproute2? Is there functionality that
> is missing and e.g. not documented at http://baturin.org/docs/iproute2/ ?

I might be wrong, but I _think_ iproute2 focus is to 'replace' ifconfig and route tools.
ethtool on the other hand is more config/stat/test tool. It allows to show stats, configure features on adapters, check firmware (and even update it sometimes), and do link tests.

Cheers,

Guilherme

[Nish Aravamudan (nacc)]

@IBM: As Christian asked in c#1, I would like clarification for what environment is being used where ethtool is not present?

Both the server ISO and the cloud images should have ethtool installed by default. So are you asking for ethtool to be present in the net installer?

[Nish Aravamudan (nacc)]

@IBM, apologies, I misread c#2. So this is really about providing ethtool in the netboot installer image?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-05-26 10:50 EDT-------
(In reply to comment #14)
> @IBM: As Christian asked in c#1, I would like clarification for what
> environment is being used where ethtool is not present?
>
> Both the server ISO and the cloud images should have ethtool installed by
> default. So are you asking for ethtool to be present in the net installer?
>
> @IBM, apologies, I misread c#2. So this is really about providing ethtool in
> the netboot installer image?

No problem nacc! Well, let me clarify:

1) I started a netboot installer and complete the installation to the system;

2) I booted the system, and ethtool wasn't present. But..I had an issue with network that might have been circumvented with ethtool (i40e driver stuff).

So, the request here is that netboot installer *does install* the ethtool by default. Would be a good bonus to get the ethtool on the installer itself (I know this installer is minimal, but I think ethtool is a small tool).

Rationale: ethtool + iproute2 are the "Swiss Army knife" for networking configuration and troubleshooting...it'd be nice to get them both installed as default (and if it's not asking too much, be present in installers too heheh). iproute2 already is!

Any questions, please let me know. Also, I'm on IRC as "gpiccoli".

Thanks,

Guilherme

[Nish Aravamudan (nacc)]

On Fri, May 26, 2017 at 7:59 AM, bugproxy <email address hidden> wrote:
> ------- Comment From <email address hidden> 2017-05-26 10:50 EDT-------
> (In reply to comment #14)
>> @IBM: As Christian asked in c#1, I would like clarification for what
>> environment is being used where ethtool is not present?
>>
>> Both the server ISO and the cloud images should have ethtool installed by
>> default. So are you asking for ethtool to be present in the net installer?
>>
>> @IBM, apologies, I misread c#2. So this is really about providing ethtool in
>> the netboot installer image?
>
> No problem nacc! Well, let me clarify:
>
> 1) I started a netboot installer and complete the installation to the
> system;

Netboot installation of what? Ubuntu server?

> 2) I booted the system, and ethtool wasn't present. But..I had an issue
> with network that might have been circumvented with ethtool (i40e driver
> stuff).

So this isn't (necessarily) about the netboot installer (or any
installer) environment, but that in all *installed* environments
(presumably server/cloud only), ethtool is available. Is that
accurate?

> So, the request here is that netboot installer *does install* the
> ethtool by default. Would be a good bonus to get the ethtool on the
> installer itself (I know this installer is minimal, but I think ethtool
> is a small tool).
>
> Rationale: ethtool + iproute2 are the "Swiss Army knife" for networking
> configuration and troubleshooting...it'd be nice to get them both
> installed as default (and if it's not asking too much, be present in
> installers too heheh). iproute2 already is!

And then there is a secondary request for adding it to the installer
environment itself.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-06-06 12:26 EDT-------
Canonical,

Guilherme is currently on vacation and he will be back next week to continue this discussion.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-06-13 15:33 EDT-------
Thanks Breno! I'm back from vacation now.

Today I discussed with Nish (nacc) on freenode in order to speedup things a bit.
Our reasoning:

i) Seed file shows ethtool *should be* installed in all Ubuntu server versions, but it's not happening. I'll try 16.04.2, 16.10 and 17.04 (netboot versions) to recall which of them has the issue.

ii) I will check also if ethtool is present during netboot installer. In case it's not, we can start discussing the addition - this discussion will be more complex due to the fact netboot installation should be really tiny.

Thanks,

Guilherme

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-06-22 10:44 EDT-------
Hi folks, I just tested multiple versions of Ubuntu, and will paste a matrix with the results.

ethtool is present?
|During installer |On distro, after first boot
16.04.2 (ISO) | NO | YES
16.04.3 (beta installer/netboot) | NO | NO
17.04 (netboot) | NO | NO
17.10 (beta installer/netboot) | NO | NO

So, the only case in which we have ethtool automatic installed is when using ISO.
Also, almost before finishing installation, the installer request input so the user can choose additional stuff to be installed. I tested with 17.10 beta netboot installer, and if I mark "Basic Ubuntu server", it does install ethtool.
So, seems a matter of change ethtool from "Basic Ubuntu server" to a core package ("standard system utilities", which is marked by default).
--

Also, if possible, I'd like to start a discussion of adding ethtool on installers. Specially on netboot installer, in which all we have is a NIC and then we need to grab everything else from network...we need the tool in order to be able to troubleshoot network issues, it's pretty fundamental in my opinion, because in netboot installers, if network fails, no way to go further. The size of binary is 387 KBytes, and it's pretty self-contained (no special lib needed, no additional files other than documentation...).

Let me know if you want another Launchpad to discuss installer presence of ethtool...opinions are much appreciated.

Thanks,

Guilherme

[Robie Basak (racb)]

Do we have a list of what ethtool gives us that iproute2 cannot do, please?

[Nish Aravamudan (nacc)]

As far as I know, and I would like IBM to be explicit about it as well, but iproute2 does not provide any ethernet device manipulation stuff. That's what ethtool provides -- modifying device settings (presumably to workaround firmware bugs or other such, during bringup).

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-06-22 12:46 EDT-------
(In reply to comment #21)
> As far as I know, and I would like IBM to be explicit about it as well, but
> iproute2 does not provide any ethernet device manipulation stuff. That's
> what ethtool provides -- modifying device settings (presumably to workaround
> firmware bugs or other such, during bringup).

Exactly this Nish.

ethtool is a modern generic approach to deal with NICs. One can enable/disable features like TSO/GSO/GRO, dump adapter status/registers, perform tests, show NIC statistics and a variety of configurations. iproute2 is focused on generic IP stack handling, whereas ethtool focus on NICs.

Also, drivers can define command interfaces, allowing ethtool to perform specific activities. Like i40e (the reason of this request for Canonical): they changed their RX path, and kept the legacy (and stable) RX code on driver, to be switched via ethtool.

If you take a look in netdev kernel list, you'll see almost all debugfs parameter proposals from drivers are criticized in favor of using ethtool approaches, like i40e did. It is a *really* important/essential tool to manage network cards.

[Nish Aravamudan (nacc)]

On Thu, Jun 22, 2017 at 7:49 AM, bugproxy <email address hidden> wrote:
> ------- Comment From <email address hidden> 2017-06-22 10:44 EDT-------
> Hi folks, I just tested multiple versions of Ubuntu, and will paste a matrix with the results.
>
> ethtool is present?
> |During installer |On distro, after first boot
> 16.04.2 (ISO) | NO | YES
> 16.04.3 (beta installer/netboot) | NO | NO
> 17.04 (netboot) | NO | NO
> 17.10 (beta installer/netboot) | NO | NO

For the three netboot cases, what is your source for the netboot
image? URL, etc.

[Nish Aravamudan (nacc)]

On Thu, Jun 22, 2017 at 10:23 AM, Nish Aravamudan
<email address hidden> wrote:
> On Thu, Jun 22, 2017 at 7:49 AM, bugproxy <email address hidden> wrote:
>> ------- Comment From <email address hidden> 2017-06-22 10:44 EDT-------
>> Hi folks, I just tested multiple versions of Ubuntu, and will paste a matrix with the results.
>>
>> ethtool is present?
>> |During installer |On distro, after first boot
>> 16.04.2 (ISO) | NO | YES
>> 16.04.3 (beta installer/netboot) | NO | NO
>> 17.04 (netboot) | NO | NO
>> 17.10 (beta installer/netboot) | NO | NO
>
> For the three netboot cases, what is your source for the netboot
> image? URL, etc.

I think this might be the disconnect -- the netboot image is a minimal
image, agnostic to installation tasks (it just gets you into the
installer environment). It does *not*, in contrast to the Ubuntu
Server ISO from releases.ubuntu.com, have any tasks pre-selected
specifically for server.

So, if you want to install "Ubuntu Server" via netboot, you must
select the "Basic Ubuntu Server" task as you reported (I think) (you
might also be able to preseed this, if you want to specifically test
this always).

Otherwise the 'seeded' entries for server will not be installed.

Now, this leads to the other issue (as I mentioned in a prior comment)
-- that you might also need ethtool itself in the minimal netboot
environment, in order to set some hardware values. I think that
decision needs to be made by the Foundations folks who are watching
this bug (Steve?).

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-06-22 13:46 EDT-------
nacc, the URLs are:

16.04.3 (beta) -> http://ppa.launchpad.net/breno-leitao/di/ubuntu/dists/xenial/main/installer-ppc64el/current/images/hwe-netboot/ubuntu-installer/ppc64el/

17.04 -> http://ports.ubuntu.com/ubuntu-ports/dists/zesty/main/installer-ppc64el/current/images/netboot/ubuntu-installer/ppc64el/

17.10 (beta) -> http://ports.ubuntu.com/ubuntu-ports/dists/artful/main/installer-ppc64el/current/images/netboot/ubuntu-installer/ppc64el/

We can test with more netboot installations if required, but I guess worth you folks to take a look in other archs too...

Thanks,

Guilherme

[Nish Aravamudan (nacc)]

On Thu, Jun 22, 2017 at 10:48 AM, bugproxy <email address hidden> wrote:
> ------- Comment From <email address hidden> 2017-06-22 13:46 EDT-------
> nacc, the URLs are:
>
> 16.04.3 (beta) -> http://ppa.launchpad.net/breno-
> leitao/di/ubuntu/dists/xenial/main/installer-ppc64el/current/images/hwe-
> netboot/ubuntu-installer/ppc64el/
>
> 17.04 -> http://ports.ubuntu.com/ubuntu-ports/dists/zesty/main
> /installer-ppc64el/current/images/netboot/ubuntu-installer/ppc64el/
>
> 17.10 (beta) -> http://ports.ubuntu.com/ubuntu-ports/dists/artful/main
> /installer-ppc64el/current/images/netboot/ubuntu-installer/ppc64el/
>
> We can test with more netboot installations if required, but I guess
> worth you folks to take a look in other archs too...

Thanks, quick summary from the #ubuntu-devel IRC discussion: the
netboot installer is a minimal image (not server specific), not
intended to have every package installed in it. The premise is,
presuming successful netboot in the first place, you then can install
whatever packages you need, either via preseed or via manual
selection.

Colin Watson pointed out the d-i environment has an ethtool-lite, and
perhaps the right choice is to extend the functionality of that tool
to support whatever is needed from ethtool itself, rather than adding
ethtool to the d-i environment.

[Dimitri John Ledkov (xnox)]

closing LP bug with status Opinion.