Ubuntu Server 20.04.3 autoinstall - User not available until reboot

[Flauschie]

As per documentation:
 - late-commands
 -- Shell commands to run after the install has completed successfully and any updates and packages installed, just before the system reboots.

Doesn’t seem to be the case. At least during late-commands I would expect the system to be finished as per description, but i.e. the defined user in identity isn’t even in `/target/etc/passwd` nor in `/etc/passwd`. Caused my late-command script to fail.

The specified user(s) in the `identity` or `user-data` section is/are not applied until the final reboot happens. Therefore, late-command is not being run in a complete state of the install!

How to check? Watch both passwd or group until the installer finishes. Nothing happens there. On reboot everything is suddenly there.

This prevents my late-command scripts to fail, `in-target target=/target`, i.e. `su: user USER does not exist`.

Manually adding users in late-command using `useradd` causes this user to be id 1000 and every other following counted up. Furthermore those users added by the installer are not being able to login then because their home directory belongs to `root:root`.

[Michael Hudson-Doyle (mwhudson)]

Yes, this is how it works, the user is created by cloud-init on first boot. If you need to run commands as that user, it might make sense to also run those commands at first boot via user-data?

[Flauschie (flauschie)]

Ah, I see, though it doesn't make sense. If cloud-init does the job the installer did not finish his work in my opinion. Either the system is ready or not.

Since you consider the system as installed at the last stage, late-commands, there is probably nothing I can do about it. But could you give me a hint into the right direction then? I read about commands to be executed as first boot but for my life I cannot find it. Is it bootcmd or runcmd but those are not in the user-data section? Or is the term user-data used here for two different things?

EDIT: Alright, I tried the runcmd solution with a simple test command

```
autoinstall:
  user-data:
    runcmd:
      - su $USER -c "ssh-keygen -t rsa -q -f /home/$USER/.ssh/id_rsa -N ''"
```

Result: Saving key "/home/USERNAME/.ssh/id_rsa" failed: Permission denied".

So, even using the recommended command it won't work since even as root trying to run the command as the user (I guess runcmd is being run as root) there is a permission issue. Probably due to the problem in my opening post about the directory not owned by the user at the time I guess.

EDIT2: Alright. I made a workaround since it simply doesn't work no matter what I tried

```
autoinstall:
  user-data:
    write_files:
      - path: /run/tmpfiles.d/script.sh
        permissions: root
        owner: root
        content: |
          my content
    runcmd:
      - /bin7bash /run/tmpfiles.d/script.sh
```

I also defined two users, where the first user in the list (*alphabetical ordered by cloud-init) has wrong rights, namely `root:root` set for his home directory and therefore cannot login... Did I do something wrong or should I report a bug for cloud-init?

```
autoinstall:
  user-data:
    users:
      - name: userone
        shell: /bin/bash
        lock-passwd: false
        passwd: hash
        groups: [one, two, three]
      - name: usertwo
        shell: /bin/bash
        lock-passwd: false
        passwd: hash
        groups: [one, two, three]
```

EDIT3: I leave this in here for others to see. I got the permission issue resolved. This seems to be because during the late-commands I run a custom script which downloads files into the users directory. This causes the directory to be owned by root: for whatever reason. I am looking for another way to provide the files since I need to have them ready in case the machine is started netless on first boot.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Nunya Business (nunya182739)]

IMO this is certainly a bug, and the comment that users are added at first boot by cloud-init rather than "now" while correct, seems like a poor design decision. I agree that once the automated install is done, the install should be complete. There should not be any installation tasks left to perform on the next boot. If there are then, IMHO, the installation isn't finished.

I have a similar situation to @Flauschie but his workaround will not work for me.

In my particular case I am building packer images for VMWare, and those images need to have a few users preinstalled with their SSH keys. Since I don't want packer to have to deal with rebooting the VM for the sole purpose of allowing it to do first-boot installation tasks, I am forced to just start experimenting with even more detailed "late-commands" statements to manually add the users to the autoinstall target, and entirely forego using the autoinstaller to add *any* users.

Simply completing the installation without requiring a "first boot" would address both of our issues rather than requiring two different workarounds to the same problem.

[Siegfried (o-gott-wir-werden-alle-sterben)]

In my usecase, I want to install a machine with a single user and change their shell to zsh. The relevant part of my user-data.yml looks like this:
autoinstall:
  identity:
    hostname: UbuntuLatest
    password: <hashed>
    username: test
  late-commands:
    - echo "<mounting the nfs and cifs shares>" >> /target/etc/fstab
    - # and other things to prepare the system like configuring gnome etc.
  user-data:
    runcmd:
      - sed -i 's#/bin/bash#/usr/bin/zsh#g' /etc/passwd

I create the user in the identity section and prepare the system in the late-commands section just as described in the docs: ubuntu.com/server/docs/install/autoinstall-reference

But the user does not exist, so I cannot change the shell accordingly, I have to add a line in the poorly documented user-data section. The splitting of the system preparations into two different sections makes this error prone. So I agree that this is a bug.