list index out of range

[sns]

after clicking Next on chroot step, it hangs. please see log attached. thank you.

[Cubic PPA (cubic-wizard)]

Thanks the bug the detailed log.

[sns (stevenospam)]

you're welcome. let me know if i can be of assistance testing anything else.

[Cubic PPA (cubic-wizard)]

There seem to be two issues here. (If you only encountered one of this issues, you would not have experienced this particular bug).

ISSUE #1:
---------

Cubic cannot identify the version of the initrd (named "initrd") file on the original ISO you are trying to customize.

ISSUE #2:
---------

Cubic does not find a vmlinuz or initrd file in your customized file system.

[Cubic PPA (cubic-wizard)]

ISSUE #1:
---------

Cubic identifies that the vmlinuz (named "vmlinu") on the original ISO is version 5.3.0-18.
Then Cubic tries to identify the version of the initrd (named "initrd")file on the original ISO.
However, for some reason on your system, the version of initrd can not be determined from the file.
Therefore, Cubic does not associate these two files with each other (to list them on a single row on the Kernels tab) even though they obviously belong together. As a result these two files are ignored by Cubic.

When I tested ubuntu-mate-19.10-desktop-amd64.iso on my system, Cubic correctly identified the initrd version:
    • The initrd version is................ 5.3.0-18

However in your logs, I see
    • The initrd version is................ 0.0.0-0

RESOLUTION:
-----------

If there is only vmlinuz and one initrd file in a directory, and Cubic is only able to determine the version of one of them, Cubic should assume that both files have the same version.

[Cubic PPA (cubic-wizard)]

ISSUE #2:
---------

Cubic does not find a vmlinuz or initrd file in your customized file system. These would be the files you see in your /boot directory ~inside~ Cubic's chroot environment. I noticed something "odd" about the 19.10 live CD from Ubuntu: there are broken links in /boot pointing to non-existent vmlinuz and initrd files.

Here is the structure of /boot

    boot
    ├── config-5.3.0-18-generic
    ├── grub
    │   ├── gfxblacklist.txt
    │   └── unicode.pf2
    ├── initrd.img -> initrd.img-5.3.0-18-generic [BROKEN LINK!]
    ├── initrd.img.old -> initrd.img-5.3.0-18-generic [BROKEN LINK!]
    ├── memtest86+.bin
    ├── memtest86+.elf
    ├── memtest86+_multiboot.bin
    ├── System.map-5.3.0-18-generic
    ├── vmlinuz -> vmlinuz-5.3.0-18-generic [BROKEN LINK!]
    └── vmlinuz.old -> vmlinuz-5.3.0-18-generic [BROKEN LINK!]

As you can see, all symlinks to the kernel files are broken.

    $ file vmlinuz
    vmlinuz: cannot open `vmlinuz' (No such file or directory)

    $ file vmlinuz.old
    vmlinuz.old: cannot open `vmlinuz.old' (No such file or directory)

    $ file initrd.img
    initrd.img: cannot open `initrd.img' (No such file or directory)

    $ file initrd.img.old
    initrd.img.old: cannot open `initrd.img.old' (No such file or directory)

According to...

    https://packages.ubuntu.com/eoan/amd64/linux-image-5.3.0-18-generic/filelist

...the file "/boot/linux-headers-5.3.0-18" should be included in package linux-headers-5.3.0-18.

Surprisingly, `dpkg -l` shows that linux-headers-5.3.0-18 is actually installed.

    $ dpkg -l linux-image-5.3.0-18-generic
    ii linux-image-5.3.0-18-generic 5.3.0-18.19+1

RESOLUTION:
-----------

This is an issue with the original ISO, because past versions of Ubuntu did not have this problem. Because the ISO boots using the kernel (vmlinuz and initrd) on the ISO, and not the kernel in the compressed Linux file system (the chroot environment in Cubic), the ISO boots up just fine without these files. (The correct kernel files are probably eventually copied to /boot during installation, so the end user never notices).

As a result, there are no code changes we can do in cubic that can directly resolve ISSUE #2, but there is something you can do, which I will post in a separate comment.

[Cubic PPA (cubic-wizard)]

What can I do now, while I wait for a bug fix (for issue #1)?

You can do something yourself to address issue #2...

REMEDY FOR ISSUE #2:
--------------------

Simply install the missing files. You can install files for kernel 5.3.0-18 (see A), which comes with 19.10, or you can install a newer kernel (see B).

(A) In Cubic's chroot, reinstall the packages with missing files:

$ sudo apt update
$ sudo apt install --reinstall linux-headers-5.3.0-18 linux-headers-5.3.0-18-generic linux-image-5.3.0-18-generic

...OR...

(B) Alternatively, you can install the latest available Linux kernel.

$ sudo apt update
$ sudo apt install linux-headers linux-headers-generic linux-image-generic

Either of these approaches will add the missing files to /boot and Cubic should be able to pick these up and display the corresponding kernel files (vmlinuz and initrd) on the Kernels tab.

[Cubic PPA (cubic-wizard)]

SNS,

If the remedy in comment 7 doesn't resolve your problem, then we will need to do a bit more work.

I suspect there may be something on your system (perhaps a missing package or library?) that is preventing Cubic from examining the inird version, because of this error in your logs...

    Encountered exception while getting
    initrd version name from file
    contents............................... cpio: premature end of archive

[sns (stevenospam)]

Thank you.

I will do a series of tests.

Test 1)

I substituted kubuntu-18.04.3-desktop-amd64.iso for the ubuntu-mate-19.10-desktop-amd64, and made no other updates to the host.

This time it finished OK, however it produced only a 96MB .iso file, which unsurprisingly fails to boot with a "unable to find medium containing a live filesystem".

Logs attached.

Future tests I will do:

2) Apply your suggested Remedy for Issue #2 (A) or (B) to the current host, and run it against the original ubuntu-mate-19.10-desktop-amd64.iso

3) Try cubic on a new clean kubuntu-18.04.3-desktop-amd64 host against the ubuntu-mate-19.10-desktop-amd64.iso
   If I understand you correctly, even though mate-19.10 is missing some files, because both Issues need to be present for this failure mode, the expectation is this should work.

4) Try cubic on a new clean kubuntu-18.04.3-desktop-amd64 host against the kubuntu-18.04.3-desktop-amd64.
   Presumably, this should work.

Thanks for your help.

[Josh (slimxwhitman)]

When you say clicking Next after the chroot step, is this when you are trying to create a new iso based on the changes you just made? as that is the issue I am having, I can't create a new iso, and I also see no kernel options after I click next.

[Cubic PPA (cubic-wizard)]

Josh,

    Please try the suggestion in comment #7, and let us know your results.

SNS, Josh,

    Would you please share some information about the systems you are running Cubic in?

    For example...

    Which Distro are you running Cubic in?
    Are you running a a virtual machine; if so how much virtual disk space?
    If on actual hardware, how much space remaining on the disk for your Cubic project?
    Are you running 32 bit or 64 bit architecture?

[Cubic PPA (cubic-wizard)]

Josh,

Are you actually able to get to the next page in Cubic when you click on Next on the chroot / terminal page?

Technically, if Cubic can't list Kernels (on the Kernels tab on the Options* page), you shouldn't be able to get to the Options page at all.

So please clarify that?

(*The Options page is the one the four tabs on it).

[Cubic PPA (cubic-wizard)]

I just installed Ubuntu Mate 19.10 x64 into a VirtualBox virtual machine w/25 GB disk space. I then installed Cubic, and created a new project to customize "ubuntu-mate-19.10-desktop-amd64.iso".

I did confirm that the vmlinuz and initrd files in /boot are broken symlinks (per issue #2).

However in my test, with regard to issue #1, Cubic had no problem finding the original vmlinuz and initrd files located at

    <CUBIC PROJECT>/original-iso-mount/casper

(Make sure that the original ISO remains mounted while you are running Cubic. Don't manually unmount it; Cubic will unmount it when it done using the original ISO).

Cubic was able to determine the correct version of the initrd file...

  Create initrd details list
    • Search directory..................... /home/user01/Temp/Ubuntu_19.10/squas
                                            hfs-root/boot
    • Number of initrd files found......... 0
  Create initrd details list
    • Search directory..................... /home/user01/Temp/Ubuntu_19.10/origi
                                            nal-iso-mount/casper
    • Number of initrd files found......... 1
    Get initrd version name from file
    name................................... /home/user01/Temp/Ubuntu_19.10/origi
                                            nal-iso-mount/casper/initrd
    Get initrd version name from file
    contents............................... /home/user01/Temp/Ubuntu_19.10/origi
                                            nal-iso-mount/casper/initrd
    • The version name is.................. 5.3.0-18
    • The initrd version is................ 5.3.0-18

[sns (stevenospam)]

Josh:
Yes your problem was similar to mine.

CubicPPA:

Thank you. Your software works great. I can confirm that your Issue 2(A) fix works perfectly.

As for Issue 1, I think you are right, it was disk space. It is an Ubuntu 18.04.1 VM, and the data drive was out of space, and I increased it and everything is good.

I have a question about root access, i notice in fix 2A above you said to prefix sudo onto the apt-get. Aren't we already root in the chroot environment and is there any way to not be root?

The reason I ask is that while I was chrooted in, i installed an app which installed some python modules to /root/.local/lib/python3.7/site-packages/* since i was root. But in the ISO this app won't work unless i launch it from a root console. Is there a way to install apps in the chrooted environment as non root?

Is there a better forum to ask other questions regarding this software?

Thank you, your software is really nice.

[Cubic PPA (cubic-wizard)]

SNS,

I'm glad it's working now.

You are right, I should not have included "sudo" in the commands. In chroot, you are already root, and there is no need to add sudo. (If you do include it, it has no effect, and the command runs the same).

Regarding your second question, see if your app has a mechanism or option for installing "globally" rather than "locally". Usually, these apps install themselves to "/opt" or "/use/share" insted of the user's current home directory.

If that option is not available, and need files available for all users, you can copy them to the /etc/skel folder, and they will be automatically copied to the new home folder for EVERY new user you create when running your customized ISO. As long as none of the files in .local/lib/python3.7/site-packages/* reference a user id or a user's home path (in this case "/root"), it should work fine when copied to /etc/skel.

Usually, questions like this can be asked by clicking on "Answers" at the top of this page (https://answers.launchpad.net/cubic).

[sns (stevenospam)]

Thank you. This program and your fantastic support are brilliant.

[Cubic PPA (cubic-wizard)]

SNS, Josh,

I just checked the daily builds of Focal Fossa (Ubuntu 20.04), and I noticed that vmlinuz and initrd are also broken symlinks in that release, as well.

If this is not fixed, issue #2 above will continue to be a problem in Cubic for future Ubuntu versions.

I've opened Bug #1860495 against Ubuntu 20.04 to correct this.

It would be great if you would mark yourselves impacted, so that this issue gets attention before the Live CD is released?

(See https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1860495)

[Josh (slimxwhitman)]

System I'm running cubic from:
Desktop - Ubuntu 16.04, 4.15.0-74-generic kernel.

Iso I'm editing: Ubuntu 16.04, kernel 4.15.0-74-generic

I can get to the options page (4 tabs) I have tried the suggestion of re-installing the kernel but still have no options in my iso boot kernel tab.

Thanks,
Josh

[Josh (slimxwhitman)]

Also I just tried from a brand new downloaded iso of 16.04 from ubuntu and I have the exact same issue. No ISO Boot Kernel listed after I did a simple apt-get update apt-get upgrade. Only did this as a test.

I am running version 2020.01-60-release~202001191802~ubuntu16.04.1 of cubic.

[Cubic PPA (cubic-wizard)]

Josh,

OK. Thanks for that info.
I'll check 16.04 and see what the issue is.

[Cubic PPA (cubic-wizard)]

Josh,

I was able to reproduce your problem.

The bug you are experiencing is ~different~ that the issue SNS has reported in this bug, because you are able to get to the Options page.

Your problem occurs when you click the "Generate" button on the Options page.

Would you please open a new bug report?

I will fix your bug under the other bug report.

[Josh (slimxwhitman)]

Created new bug > https://bugs.launchpad.net/cubic/+bug/1860682

[Cubic PPA (cubic-wizard)]

Bug #1860682, "ISO Boot Kernel Empty after clicking generate Edit" was opened to address issue described in comment number 18 (https://bugs.launchpad.net/cubic/+bug/1860345/comments/18).

[Cubic PPA (cubic-wizard)]

Lowering the priority of this bug, since the original reporter's issue was resolved by increasing the disk space available to Cubic.

Issue number 1, where the initrd version cannot be determined, seems to have been caused by low disk space.

This bug will be kept open to implement the change described in comment umber 5. (See https://bugs.launchpad.net/cubic/+bug/1860345/comments/5).

[Cubic PPA (cubic-wizard)]

The solution to ISSUE #1 must take into account the following scenarios:

1. Any directory that only has one vmlinuz file and only one initrd file. (It is only reasonable to assume vmlinuz and initrd versions match when there is only one of each in a directory).
2. The version of vmlinuz can be determined, and the version of initrd can not be determined.
3. The version of vmlinuz can not be determined, and the version of initrd can be determined.
4. The version of both vmlinuz and initrd can not be determined.
5. Symlinks to directories or symlinks to vmlinuz and initrd files.
6. Relative symlinks. These work fine in chroot and outside chroot (where Cubic runs).
7. Direct symlinks to files in sub-directories of /boot.
8. Direct symlinks to files outside /boot. (A symlink in /boot that points to a file in /; for example: /boot/vmlinuz --> /some_dir_in_root/vmlinuz). These symlinks are valid inside chroot but appear as broken links to Cubic (outside chroot).

[Cubic PPA (cubic-wizard)]

Fix for Issue #1 released to Development branch, revision 213.

[Cubic PPA (cubic-wizard)]

Fix released to Development branch 214.
Fix released to Release branch 62.