identify brightbox platform to enable Ec2 datasource

[Scott Moser]

Brightbox provides an EC2 Metadata service lookalike, and that is how cloud-init
gets metadata on their platform.

In an effort to avoid polling metadata services in cloud-init we will disable
attempts to reach the MD without positive identification of the cloud.

We need to be able to positively identify that the host platform is
brightbox so we can safely assume there will be a metadata service there.

The easiest thing is for Brightbox to put something in dmi tables that identify
their platform. See bug 1660385 for more information and for how Amazon does
this.

Related bugs
 bug 1660385: Alert user of Ec2 Datasource on lookalike cloud
 bug 1661797: identify lxd-nova platform to enable Openstack datasource
 bug 1661693: identify brightbox platform to enable Ec2 datasource
 bug 1663304: identify openstack kvm platform on arm64
 bug 1663315: identify openstack kvm platform on ppc64

[Neil Wilson (neil-aldur)]

There isn't a standard way of identifying cloud platforms from what I can see. Are you going to be maintaining a list of the myriad of different mechanisms?

What's the driver for doing this? Is there a newer way of getting metadata into Ubuntu?

[Scott Moser (smoser)]

Neil,

You're right, there is not a standard way of identifying cloud platforms.
One way of doing such is as Openstack Nova does.
I think this is a reasonable "best practice".

Amazon shows what they are doing at
 http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/identify_ec2_instances.html

To show what openstack does:
$ cat /sys/class/dmi/id/product_name
OpenStack Nova
$ sudo dmidecode --string=system-product-name
OpenStack Nova

From outside, virsh xml contains:
  <sysinfo type='smbios'>
    <system>
      <entry name='manufacturer'>OpenStack Foundation</entry>
      <entry name='product'>OpenStack Nova</entry>
      <entry name='version'>13.1.1</entry>
      <entry name='serial'>fb14e3a3-bff8-4e1a-8d27-ee2a2414d822</entry>
      <entry name='uuid'>0c0fb83e-a83f-49c2-b483-db405fbdc8ee</entry>
      <entry name='family'>Virtual Machine</entry>
    </system>

And qemu cmdline contains:
  -smbios "type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=12.0.3,serial=fb14e3a3-bff8-4e1a-8d27-ee2a2414d822,uuid=bf7f3844-42d3-4bcf-a58c-a33b0c12cb28,family=Virtual Machine"

The driver for doing this is:
a.) to improve boot speed of Ubuntu images by quickly discarding datasources that are not expected to be there.
b.) to utilmately entirely disable cloud-init if no datasource would be found.
c.) to avoid attempting to access network resources that may not be present. That can result in timeouts or other unexpected behavior.

[Scott Moser (smoser)]

Any thoughts here? Any help I can offer?

[Neil Wilson (neil-aldur)]

Scott,

Apologies. Been knee deep in kernel bugs this week and dropped the ball with this one.

We're thinking of setting the family in DMI table 1 thus:

  <os>
    <type arch='x86_64'>hvm</type>
    <boot dev='hd'/>
    <smbios mode='sysinfo'/>
  </os>
  <sysinfo type='smbios'>
    <system>
      <entry name='family'>Brightbox Cloud</entry>
    </system>
  </sysinfo>

Which gives:

Handle 0x0100, DMI type 1, 27 bytes
System Information
    Manufacturer: Red Hat
    Product Name: KVM
    Version: RHEL 7.0.0 PC (i440FX + PIIX, 1996)
    Serial Number: Not Specified
    UUID: B4BB19C4-8F2A-4D60-AED9-22CBE259CFF9
    Wake-up Type: Power Switch
    SKU Number: Not Specified
    Family: Brightbox Cloud

Does that work for you?

[Scott Moser (smoser)]

Neil,
I could work with family if I had too. Its less than ideal.
The kernel populates some from the smbios into /sys, but apparently 'Family' is not one of those.

I launched a system with xml like yours above, which renders qemu cmdline like:
 -smbios "type=1,manufacturer=Smoser Foo,family=Brightbox Cloud"

From the inside, dmidecode *does* show me the family as you show above (sudo dmidecode --type system).
However, family is not available with either:
  dmidecode --string=SOME_ARGUMENT
nor is it available in /sys/class/dmi/id/ . So I'd have to run dmidecode and parse its output or otherwise implement a dmi/smbios reader.

An ideal value is something that the kernel exposes, that you can see with:
  $ sudo grep -r . /sys/class/dmi/id

[Scott Moser (smoser)]

Just for reference, the product name (libvirt <entry name='product'>, /sys/class/dmi/id/product_name) is used for this purpose on:
  - Google Compute Engine
  - CloudStack
  - CloudSigma
  - OpenStack Nova
  - SmartOS (Joyent cloud)

So I'd like to suggest that that is the working "best practice".

[Neil Wilson (neil-aldur)]

But not on AWS or Digital Ocean?

[Dimitri John Ledkov (xnox)]

Hi,

On 22 February 2017 at 17:54, Neil Wilson <email address hidden> wrote:
> But not on AWS or Digital Ocean?
>

On Digital Ocean:
$ cat /sys/class/dmi/id/product_name
Droplet

I don't have access to AWS to check that.

Regards,

Dimitri.

[Neil Wilson (neil-aldur)]

systemd code on RHEL 7 looks like it checks the product name.

https://github.com/lnykryn/systemd-rhel/commit/795d7cadb7b49ae11e2544ce325779f8d5ec7526

Similarly virt-tools on RHEL looks for KVM in the product name

http://www.redhat.com/archives/virt-tools-list/2015-October/msg00094.html

So there are problems with the 'product-name' approach from our point of view.

[John Leach (johnleach)]

puppet also uses the product name to provide it's is_virtual fact:

https://github.com/puppetlabs/facter/blob/b0726f54a5e8276ce756a126eb1fdd5b4fb08609/lib/src/facts/resolvers/virtualization_resolver.cc#L74-L94

It knows about "KVM" and a few others, though it at least looks for KVM anywhere in the product name. But it currently won't understand about the above names.

This situation seems like a bit of a mess - no standard way of providing this info, everyone doing their own thing.

[Neil Wilson (neil-aldur)]

The system serial number is empty under KVM and exposed in the sysfs, so I propose putting the fqdn of the server into that field.

That way you can pick up a Brightbox cloud server fairly easily:

with open('/sys/devices/virtual/dmi/id/product_serial', 'r') as content_file:
    content = content_file.read()

if content.rstrip().endswith(".brightbox.com"):
    print("Brightbox VM")
else:
    print("Not a Brightbox VM")

[Scott Moser (smoser)]

For further reference, DigitalOcean sets sys_vendor to 'DigitalOcean'.
AWS's is described at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/identify_ec2_instances.html
its crappy as they abuse UUID, which will result in false-positive if randomly generated uuid starts with 'ec2'. Its better on kvm guests there then it is on xen in that system-serial-number and system-uuid have to be equal and start with ec2, so a false positive there is pretty unlikely.

[Scott Moser (smoser)]

Neil, i'm fine with product_serial containing 'brightbox'.
Given the fact that Google and Openstack and others use the product_name as shown above, it feels like puppet and rhel must have other ways of determining this. But I respect your interest in not breaking anything.

I'll add a check for brightbox today.

[Scott Moser (smoser)]

New zesty images with cloud-init 0.7.9-38-g83606ae-0ubuntu1 or better should now identify brightbox by looking for brightbox.com in the serial as Neil suggested in comment 11.

It does not appear that the change is made on the host side yet. I launched an instance just now and see:

$ ( cd /sys/class/dmi/id && sudo grep -r . *) 2>/dev/null
bios_date:01/01/2007
bios_vendor:Seabios
bios_version:0.5.1
chassis_type:1
chassis_vendor:Red Hat
modalias:dmi:bvnSeabios:bvr0.5.1:bd01/01/2007:svnRedHat:pnKVM:pvrRHEL6.6.0PC:cvnRedHat:ct1:cvr:
power/control:auto
power/async:disabled
power/runtime_enabled:disabled
power/runtime_active_kids:0
power/runtime_active_time:0
power/runtime_status:unsupported
power/runtime_usage:0
power/runtime_suspended_time:0
product_name:KVM
product_uuid:9D740B41-32E9-47B7-E119-2B8884E27421
product_version:RHEL 6.6.0 PC
sys_vendor:Red Hat
uevent:MODALIAS=dmi:bvnSeabios:bvr0.5.1:bd01/01/2007:svnRedHat:pnKVM:pvrRHEL6.6.0PC:cvnRedHat:ct1:cvr:

[Neil Wilson (neil-aldur)]

The serial update is now in production. Latest zesty image we have is from 15th Feb and is not picking up the network (probably as expected).

[Scott Moser (smoser)]

Verified this is working on brightbox now. Thanks for your cooperation and help, Neil.

Booted img-foxkf (com.ubuntu.cloud:daily:download/com.ubuntu.cloud.daily:server:17.04:amd64/20170228.1/disk1.img)

inside:
$ cat /etc/cloud/build.info
build_name: server
serial: 20170228.1

$ dpkg-query --show cloud-init
cloud-init 0.7.9-38-g83606ae-0ubuntu1

$ grep brightbox -i /var/log/cloud-init.log
2017-03-01 19:17:17,514 - util.py[DEBUG]: dmi data /sys/class/dmi/id/product_serial returned srv-fajvm.gb1.brightbox.com
2017-03-01 19:17:17,514 - DataSourceEc2.py[DEBUG]: strict_mode: warn, cloud_platform=Brightbox
2017-03-01 19:17:19,755 - cc_set_hostname.py[DEBUG]: Setting the hostname to srv-fajvm.gb1.brightbox.com (srv-fajvm)
2017-03-01 19:17:19,764 - cc_update_hostname.py[DEBUG]: Updating hostname to srv-fajvm.gb1.brightbox.com (srv-fajvm)

$ grep -i ec2 /run/cloud-init/ds-identify.log
/etc/cloud/cloud.cfg.d/90_dpkg.cfg set datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Ec2, CloudStack, None ]
DSLIST=NoCloud ConfigDrive OpenNebula DigitalOcean Azure AltCloud OVF MAAS GCE OpenStack CloudSigma SmartOS Ec2 CloudStack None
ec2 platform is 'Brightbox'.
check for 'Ec2' returned found
Found single datasource: Ec2

$ ( cd /sys/class/dmi/id && sudo grep -r . . ) 2>/dev/null
./product_serial:srv-fajvm.gb1.brightbox.com
./bios_vendor:Seabios
./product_version:RHEL 6.6.0 PC
./power/runtime_active_kids:0
./power/runtime_suspended_time:0
./power/runtime_enabled:disabled
./power/runtime_active_time:0
./power/control:auto
./power/async:disabled
./power/runtime_usage:0
./power/runtime_status:unsupported
./chassis_vendor:Red Hat
./modalias:dmi:bvnSeabios:bvr0.5.1:bd01/01/2007:svnRedHat:pnKVM:pvrRHEL6.6.0PC:cvnRedHat:ct1:cvr:
./product_uuid:D5BF3AE9-7D9C-2923-0B1E-B0ED967E7DF0
./bios_version:0.5.1
./sys_vendor:Red Hat
./chassis_type:1
./uevent:MODALIAS=dmi:bvnSeabios:bvr0.5.1:bd01/01/2007:svnRedHat:pnKVM:pvrRHEL6.6.0PC:cvnRedHat:ct1:cvr:
./product_name:KVM
./bios_date:01/01/2007

$ cat /var/lib/cloud/data/instance-id
srv-fajvm

[Scott Moser (smoser)]

Just a comment.
This should now be 'fix-released' in xenial, and yakkety also with 0.7.9-48-g1c795b9-0ubuntu1~16.04.1 and 0.7.9-48-g1c795b9-0ubuntu1~16.10.1 respectively.

[Scott Moser (smoser)]

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2798