text/x-shellscript doesn't run if text/cloud-boothook present

[DaveQB]

From my testing, it seems if you have a multipart user-data file that contains both a boothook and a shell script, the latter will not run. I have been testing with all sorts of user-data files and only in this setup (a boothook AND a script) do I see the shell script not run at all.

This is using the the Amazon Ubuntu 12.04.1 AMI (ami-ac4300fe)

PS
The reason I am at this point is that running "echo myhostname > /etc/hostname" in the boothook sees it overwritten by cloud-init. Even with a line like this in the boothook "sed -i 's/preserve_hostname/preserve_hostname: True/g' /etc/cloud/cloud.cfg" hence I have to add a shell script for this.

Thanks.

[Scott Moser (smoser)]

Hi,
  Thanks for taking the time to open a bug and help in making Ubuntu better.
  I'm not able to recreate the issue you're seeing. Working off the doc at https://help.ubuntu.com/community/CloudInit . I've attached the user-data that was created by write-mime-multipart, but here is what I did to try to reproduce:

# ami-9c78c0f5 = us-east-1 ubuntu-precise-12.04-amd64-server-20121026.1

$ cat my-boothook
#!/bin/sh
echo "=== $(date): running $0 $* ===" | tee -a /root/boothook.log
echo myhostname > /etc/hostname

$ cat my-userscript
#!/bin/sh
echo "=== $(date): running $0 $* ===" | tee -a /root/userscript.log

$ write-mime-multipart --output=userdata.txt \
   my-boothook:text/cloud-boothook my-userscript:text/x-shellscript

$ euca-run-instances --key=brickies --instance-type=t1.micro \
   --user-data-file=userdata.txt ami-9c78c0f5

Then, inside the instance I did:
$ sudo sh -c 'for f in /root/*.log; do echo "#$f"; cat $f; done'
#/root/boothook.log
=== Wed Dec 5 19:09:36 UTC 2012: running /var/lib/cloud/instances/i-06d52c78/boothooks/my-boothook ===
#/root/userscript.log
=== Wed Dec 5 19:09:49 UTC 2012: running /var/lib/cloud/instance/scripts/my-userscript ===

That shows that the scripts ran 13 seconds apart (one early in boot and one later). And by their names, its clear that one ran as a boothook and one as a user-script.

[Scott Moser (smoser)]

Could you give me an example that fails?

Also, just fyi,
a.) you dont have to change /etc/cloud/cloud.cfg's value of preserve_hostname. cloud-config input setting that will do the same thing as you were attempting.
b.) you can set the hostname by cloud-config input with 'hostname'.
  #cloud-config
  hostname: myhostname.my.fully.qualified.domain.com

[DaveQB (david-dward)]

Thanks for the info Scott.

To to clarify, I under you can user hostname in a cloud-config, the problem I am solving is for autoscaling launch configuration and needing a common base hostname (for config management purposes and only for humans to identify) but unique amongst them.

So the host name I am setting is:

prd-myshinnyhostname-ap-southeast-1a-$RANDOM

Which I can't see anyway to do in cloud-config unfortunately, otherwise I would stick to use that for this autoscale launch config.
I am injecting into /etc/cloud/cloud.cfg to save having to have a cloud-config file at all. Just lazy maybe, but figured it works best with how I am setting the hostname (cloud-config hostname is useless for this use case)

So back to the issue, I had the below. The boothook worked but the shell script didn't touch the file in /etc or the test file.
(the work around I have for now is to just have a boothook, and the boothook injects into /etc/rc.local the shell script and then adds a line to clean up after it has run so as to have a one-time run of the shell script. Working well)

I will try your user-data file and report back.

________________________

Content-Type: multipart/mixed; boundary="===============2371311152411390760=="
MIME-Version: 1.0

--===============2371311152411390760==
Content-Type: text/cloud-boothook; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="boothook"

#cloud-boothook
#!/bin/bash

if [ ! -f /etc/id ]
then
sed -i 's/preserve_hostname/preserve_hostname: True/g' /etc/cloud/cloud.cfg
sed -i 's/apt_preserve_sources_list/apt_preserve_sources_list: True/g' /etc/cloud/cloud.cfg
# set hostname
OLD=prd-TEST-ap-southeast-1
NEW=${OLD}-$RANDOM
echo ${NEW} > /etc/hostname
hostname ${NEW}
echo 127.0.1.2 ${NEW}.domain.edu.au ${NEW} >> /etc/hosts
echo $INSTANCE_ID > /etc/id
fi

exit 0

--===============2371311152411390760==
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="boothook-script"

#!/bin/bash -x
touch /etc/$(date +%F)
H=test
echo $H > /etc/test

[DaveQB (david-dward)]

Ok. There must be something up with my scripts then. Launching with your user-data file Scott,

root@ip-10-199-0-143:~# cat userscript.log
=== Fri Dec 7 12:01:16 EST 2012: running /var/lib/cloud/instance/scripts/my-userscript ===
root@ip-10-199-0-143:~# cat boothook.log
=== Fri Dec 7 12:01:09 EST 2012: running /var/lib/cloud/instances/i-439f5914/boothooks/my-boothook ===
root@ip-10-199-0-143:~# cat /etc/hostname
ip-10-199-0-143
root@ip-10-199-0-143:~#

Let me do some more testing with this information to further troubleshoot this.

Thanks Scott.

[Scott Moser (smoser)]

> #cloud-boothook
> #!/bin/bash
>
> if [ ! -f /etc/id ]
> then
> sed -i 's/preserve_hostname/preserve_hostname: True/g' /etc/cloud/cloud.cfg
> sed -i 's/apt_preserve_sources_list/apt_preserve_sources_list: True/g' /etc/cloud/cloud.cfg

It seems to me that the sed editing here might be creating invalid yaml or
at least confusing content in /etc/cloud/cloud.cfg, which would cause
cloud-init to fail next time it loaded this file.

Ie, i think you've now got:
 preserve_hostname: TrueORIGINAL_VALUE_HERE
 apt_preserve_sources_list: TrueORIGINAL_VALUE_HERE

one thing that might make this easier, and would have the same affect, is
just:
 echo "preserve_hostname: True" > /etc/cloud/cloud.cfg.d/preserve-hostname.cfg
 echo "apt_preserve_sources_list: True" > /etc/cloud/cloud.cfg.d/apt_preserve_sources_list

[DaveQB (david-dward)]

Yes actually I did fix that in a later boothook, but this is after I went with the all-in-one boothook (that injects into rc.local to run a shell script once network was up)

I see. So messing with the cloud.cfg broke the system. My bad.

Thanks for the time Scott and sorry for sending you on a wild goose chase.
That info on making a "preserve-hostname.cfg" and "apt_preserve_sources_list" file is brilliant!! Exactly what I need. I didn't come across this in docos. Any links to this to what I have missed reading?

Thanks again Scott.

[DaveQB (david-dward)]

Error in syntax. No bug here.

[Scott Moser (smoser)]

On Fri, 7 Dec 2012, DaveQB wrote:

> Yes actually I did fix that in a later boothook, but this is after I
> went with the all-in-one boothook (that injects into rc.local to run a
> shell script once network was up)
>
> I see. So messing with the cloud.cfg broke the system. My bad.
>
> Thanks for the time Scott and sorry for sending you on a wild goose chase.
> That info on making a "preserve-hostname.cfg" and "apt_preserve_sources_list" file is brilliant!! Exactly what I need. I didn't come across this in docos. Any links to this to what I have missed reading?

Probably not. :-(
Generally, though the config parsing goes:
 * /etc/cloud/cloud.cfg
 * /etc/cloud/cloud.cfg.d/*.cfg # in C locale sorted order
 * user-data-cloud-config , parsed in order seen.

fwiw, under bug 1023179, we're hoping to make the "merge cloud-config"
more sane. Right now, it basically only overrides. Ie, if 2 files have
'runcmd' defined, the last one wins.

[DaveQB (david-dward)]

Yeah I did stumble upon that bug when I was researching this issue last week.

Thanks for all your help Scott.