Bad storage config for two disk system

[Arturas]

Hi,

I'm facing a problem while trying to crate software raid and deploy a system.

There are two disks sda and sdb. In case if boot_flag is set at sda disk config for curtin is being generated incorrectly:

partitioning_commands:
  builtin: [curtin, block-meta, custom]
storage:
  config:
  - {grub_device: true, id: sda, model: TOSHIBA DT01ACA3, name: sda, ptable: gpt,
    serial: 44NYRR2GS, type: disk, wipe: superblock}
  - {id: sdb, model: TOSHIBA DT01ACA3, name: sdb, ptable: gpt, serial: 96E8PNTAS,
    type: disk, wipe: superblock}
  - {device: sda, id: sda-part2, name: sda-part2, number: 2, size: 998244352B, type: partition,
    uuid: f1faf741-d203-4ab5-80a1-10c333826349, wipe: superblock}
  - {device: sda, id: sda-part3, name: sda-part3, number: 3, size: 2999585865728B,
    type: partition, uuid: fc8db53a-533c-4efb-882d-627d1ae41f41, wipe: superblock}
  - {device: sdb, flag: bios_grub, id: sdb-part1, number: 1, offset: 4194304B, size: 1048576B,
    type: partition, wipe: zero}
  - devices: [sda-part2, sdb-part1]
    id: md0
    name: md0
    raidlevel: 1
    spare_devices: []
    type: raid
  - {device: sdb, id: sdb-part1, name: sdb-part1, number: 1, size: 998244352B, type: partition,
    uuid: dc0f666d-9819-467c-b0d9-589daf43a37f, wipe: superblock}
  - {device: sdb, id: sdb-part2, name: sdb-part2, number: 2, size: 2999585865728B,
    type: partition, uuid: ae8377a9-0c87-4b89-9992-8b6c78a91768, wipe: superblock}
  - devices: [sda-part3, sdb-part2]
    id: md1
    name: md1
    raidlevel: 0
    spare_devices: []
    type: raid
  - {fstype: ext4, id: md0_format, label: '', type: format, uuid: d7d3f84b-0c87-4e99-a4a5-3e4889828ab2,
    volume: md0}
  - {fstype: ext4, id: md1_format, label: '', type: format, uuid: 972b2c67-f64a-4fcc-b75f-4fadc473f7b8,
    volume: md1}
  - {device: md1_format, id: md1_mount, options: '', path: /, type: mount}
  - {device: md0_format, id: md0_mount, options: '', path: /boot, type: mount}

As you may see, in that case, since sda is first disk, it should create 1MB partition on SDA, but it creates two sdb-part1 partitions. Because of that, then curtin tries to found previous partition of SDA, it can't and deployment fails.

In case if SDB selected as boot device, maas generates valid layout.

[Arturas (arturasr)]

MAAS version: 2.2.1

[Arturas (arturasr)]

I forgot to show the actual error:

        previous partition number for 'sda-part2' found to be 'None'
        previous partition: /sys/class/block/sda/sdaNone
        An error occured handling 'sda-part2': FileNotFoundError - [Errno 2] No such file or directory: '/sys/class/block/sda/sdaNone/size'
        finish: cmd-install/stage-partitioning/builtin/cmd-block-meta: FAIL: configuring partition: sda-part2
        finish: cmd-install/stage-partitioning/builtin/cmd-block-meta: FAIL: curtin command block-meta
        Traceback (most recent call last):
          File "/curtin/curtin/commands/main.py", line 215, in main
            ret = args.func(args)
          File "/curtin/curtin/commands/block_meta.py", line 64, in block_meta
            meta_custom(args)
          File "/curtin/curtin/commands/block_meta.py", line 1089, in meta_custom
            handler(command, storage_config_dict)
          File "/curtin/curtin/commands/block_meta.py", line 471, in partition_handler
            "size"))
          File "/curtin/curtin/util.py", line 364, in load_file
            with open(path, "rb") as fp:
        FileNotFoundError: [Errno 2] No such file or directory: '/sys/class/block/sda/sdaNone/size'
        [Errno 2] No such file or directory: '/sys/class/block/sda/sdaNone/size'

Stderr: ''

[Blake Rouse (blake-rouse)]

This is very strange the YAML has:

sda-part2 and sda-part3 being created, but sda-part1 is not created, but is used by the raid configration. Really it should just have sda-part1 and sda-part2, sda-part3 should not be created.

Can you include the output of:

maas {session} boot-resources read {system-id}

[Arturas (arturasr)]

Hello, Blake,

thank you for your response. Interesting, I thought sda-part1 is necessary, since sda is GPT disk, which is bigger than 2TB, so it creates 1MB partition sda-part1 for bios_grub on it. Am I wrong here?

Is boot-resources really connected to that issue? Such command even can't take arguments like system_id. Perhaps you meant block-devices? In that case the output is:

maas arturas-maas block-devices read yshm7y
Success.
Machine-readable output follows:
[
    {
        "available_size": 0,
        "block_size": 4096,
        "used_size": 3000590401536,
        "resource_uri": "/MAAS/api/2.0/nodes/yshm7y/blockdevices/392/",
        "size": 3000592982016,
        "partitions": [
            {
                "type": "partition",
                "resource_uri": "/MAAS/api/2.0/nodes/yshm7y/blockdevices/392/partition/1103",
                "id": 1103,
                "size": 998244352,
                "device_id": 392,
                "path": "/dev/disk/by-dname/sda-part2",
                "filesystem": {
                    "fstype": "raid",
                    "label": null,
                    "uuid": "3f608740-0479-4b45-994c-6f24ecf07aa5",
                    "mount_options": null,
                    "mount_point": null
                },
                "used_for": "Active raid-1 device for md0",
                "uuid": "3f8c9c5d-a4c3-43c0-931e-5556c302e86b",
                "system_id": "yshm7y",
                "bootable": false
            },
            {
                "type": "partition",
                "resource_uri": "/MAAS/api/2.0/nodes/yshm7y/blockdevices/392/partition/1107",
                "id": 1107,
                "size": 2999585865728,
                "device_id": 392,
                "path": "/dev/disk/by-dname/sda-part3",
                "filesystem": {
                    "fstype": "raid",
                    "label": null,
                    "uuid": "7b281b90-d973-41a0-91b7-67fa8f3ce93a",
                    "mount_options": null,
                    "mount_point": null
                },
                "used_for": "Active raid-0 device for md1",
                "uuid": "ae06857e-6d7d-4b09-b9a1-738cd86362a1",
                "system_id": "yshm7y",
                "bootable": false
            }
        ],
        "serial": "44NYRR2GS",
        "id_path": "/dev/disk/by-id/wwn-0x5000039ff4db9b86",
        "system_id": "yshm7y",
        "id": 392,
        "uuid": null,
        "tags": [
            "rotary",
            "sata",
            "7200rpm"
        ],
        "path": "/dev/disk/by-dname/sda",
        "name": "sda",
        "used_for": "GPT partitioned with 2 partitions",
        "filesystem": null,
        "model": "TOSHIBA DT01ACA3",
        "type": "physical",
        "partition_table_type": "GPT"
    },
    {
        "available_size": 0,
        "block_size": 4096,
        "used_size": 3000590401536,
        "resource_uri": "/MAAS/api/2.0/nodes/yshm7y/blockdevices/393/",
        "size": 3000592982016,
        "partitions": [
            {
                "type": "partition",
                "resource_uri": "/MAAS/api/2.0/nodes/yshm7y/blockdevices/393/partition/1104",
                "id": 1104,
  ...

[Blake Rouse (blake-rouse)]

MAAS does not create the bios_grub partition if the first disk is set the GPT I would assume your system is booting UEFI. If that is not the case then you probably changed how it boots since the last time you commissioned it.

Have you tried commissioning it again?

[Arturas (arturasr)]

Hello, Blake,

nope, it's not UEFI, and I tried to commission it one more time, just to be sure, but the problem remains.

As evidence, that it's not UEFI, I can provide line from 00-maas-00-support-info:

platform:platform-framebuffer

Which I believe on uefi differs.

What else I can provide?

[Chris Sanders (chris.sanders)]

I've hit what I believe is the same bug with MAAS 2.3 and GPT on non-EFI system as well.

After a fresh commission I can deploy with the default configuration, however editing any partition on sda causes a curtin which does not contain sda-part-1 and the fails with the following error during deployment.

Deployment Error:
curtin: Installation started. (17.1-11-ga4c9636b-0ubuntu1~16.04.1)
third party drivers not installed or necessary.
An error occured handling 'sda-part2': FileNotFoundError - [Errno 2] No such file or directory: '/sys/class/block/sda/sdaNone/size'
[Errno 2] No such file or directory: '/sys/class/block/sda/sdaNone/size'
curtin: Installation failed with exception: Unexpected error while running command.
Command: ['curtin', 'block-meta', 'custom']
Exit code: 3
Reason: -
Stdout: An error occured handling 'sda-part2': FileNotFoundError - [Errno 2] No such file or directory: '/sys/class/block/sda/sdaNone/size'
        [Errno 2] No such file or directory: '/sys/class/block/sda/sdaNone/size'

Stderr: ''

After a commission the full curtin which is used is:
apt:
  preserve_sources_list: false
  primary:
  - arches:
    - default
    uri: http://archive.ubuntu.com/ubuntu
  security:
  - arches:
    - default
    uri: http://archive.ubuntu.com/ubuntu
cloudconfig:
  maas-cloud-config:
    content: "#cloud-config\ndatasource:\n MAAS: {consumer_key: 9VMunpfAby3a3dqT4G,\
      \ metadata_url: 'http://192.168.0.5:5240/MAAS/metadata/',\n token_key: NETfjCsUvQtJWxQF5w,\
      \ token_secret: MkPV8MqsgjdYKTDumSJCZtcz2KTFFPQY}\n"
    path: /etc/cloud/cloud.cfg.d/90_maas_cloud_config.cfg
  maas-datasource:
    content: 'datasource_list: [ MAAS ]'
    path: /etc/cloud/cloud.cfg.d/90_maas_datasource.cfg
  maas-reporting:
    content: "#cloud-config\nreporting:\n maas: {consumer_key: 9VMunpfAby3a3dqT4G,\
      \ endpoint: 'http://192.168.0.5:5240/MAAS/metadata/status/6hbfym',\n token_key:\
      \ NETfjCsUvQtJWxQF5w, token_secret: MkPV8MqsgjdYKTDumSJCZtcz2KTFFPQY,\n type:\
      \ webhook}\n"
    path: /etc/cloud/cloud.cfg.d/90_maas_cloud_init_reporting.cfg
  maas-ubuntu-sso:
    content: '#cloud-config

      snappy: {email: admin}

      '
    path: /etc/cloud/cloud.cfg.d/90_maas_ubuntu_sso.cfg
debconf_selections:
  grub2: grub2 grub2/update_nvram boolean false
  maas: 'cloud-init cloud-init/datasources multiselect MAAS

    cloud-init cloud-init/maas-metadata-url string http://192.168.0.5:5240/MAAS/metadata/

    cloud-init cloud-init/maas-metadata-credentials string oauth_token_key=NETfjCsUvQtJWxQF5w&oauth_token_secret=MkPV8MqsgjdYKTDumSJCZtcz2KTFFPQY&oauth_consumer_key=9VMunpfAby3a3dqT4G

    cloud-init cloud-init/local-cloud-config string apt:\n preserve_sources_list:
    false\n primary:\n - arches: [default]\n uri: http://archive.ubuntu.com/ubuntu\n security:\n -
    arches: [default]\n uri: http://archive.ubuntu.com/ubuntu\napt_preserve_sources_list:
    true\nmanage_etc_hosts: false\nmanual_cache_clean: true\nreporting:\n maas: {consumer_key:
    9VMunpfAby3a3dqT4G, endpoint: ''http://192.168.0.5:5240/MAAS/metadata/status/6hbfym'',\n token_key:
    NETfjCsUvQtJWxQF5w, token_secret: MkPV8Mqs...

[Andres Rodriguez (andreserl)]

Hi There,

Is this still an issue in the latest version of MAAS (2.3.2 or 2.4)? If so, please mark this bug report as 'New'.

[Arturas (arturasr)]

Hi, Andres,

yes, the issue persists even with 2.4. It still generates config with sda-part2.

[Arturas (arturasr)]

Well, I think I should add more details on how this bug behaves on 2.4 maas. What I'm trying to do is to create Software RAID from two GPT disks (3TB).

As I understand. MaaS reserves 1st partition of SDA disk for bios_grub, so it creates partition "idx + 2", which is sda-part2, but it never creates layout for that first partition. When curtin trys to create that layout on actual drives, it can't get any info for previous partition and fails.

[Arturas (arturasr)]

One more update. Double checked the code. I tried it with 2.4.0, but it is fixed in 2.4.1:

https://bugs.launchpad.net/maas/+bug/1740442

[Arturas (arturasr)]

https://bugs.launchpad.net/maas/+bug/1740442