cloud-init does not execute userdata when /etc/rsyslog.d is missing (rsyslogd is purged)

[Jeff Webb]

I've only tried this on AWS ec2 with ubuntu 12.04 LTS (dpkg -l cloud-init shows version 0.6.3-0ubuntu1). Some details here: https://forums.aws.amazon.com/message.jspa?messageID=492656#492656

[Jeff Webb (jeff-webb-g)]

seems like https://bugzilla.redhat.com/show_bug.cgi?id=953530 might be related

[Scott Moser (smoser)]

The problem here is that /etc/init/cloud-config.conf
is 'start on (filesystem and started rsyslog)'

if rsyslog-ng is installed, then 'rsyslog' will never be 'started'. So cloud-config never starts. That is a requirement for lots of other things to run (such as handling user-data).

Its possible that some cloud-config.conf could be created to support a more generic syslog, or the lack of syslog.

rsyslog is a dependency of 'ubuntu-minimal', so its really not that crazy.

I'm open to accepting patches, but it would be considered low priority.

[Jeff Webb (jeff-webb-g)]

Hey Scott,

Thanks so much for the quick reply. What you're saying doesn't agree with what I've observed. If I
# remove rsyslog without purge
sudo apt-get -q -y remove rsyslog
sudo apt-get -q -y install syslog-ng

Userdata will be executed even though rsyslogd was not started. I checked the /etc/init/cloud-init.conf on the image I've been playing with (ami-70f96e40 if it helps) and it's 'start on mounted MOUNTPOINT=/ and stopped cloud-init-nonet'.

by the way, I may have made a false assumption about the /etc/rsyslog.d. I'm not sure that the presence of that specific directory is important, but I am sure that cloud-init behaves differently between a purge and regular remove of the rsyslog package.

- Jeff

[Scott Moser (smoser)]

cloud-config.conf != cloud-init.conf.

--purge removes the config files, of which the upstart job is one.
if this does happen to work, and the cloud-config job actually does execute, then I'd consider it a bug in upstart as upstart would have to have considered rsyslog to have 'started' which clearly cannot happen as 'exec rsyslogd' (from /etc/init/rsyslog.conf) is not going to work.

So, if remove but not purge happens to work fo ryou thats fine, but I consider that happenstance really.

I'm not opposed to adding support for "alternative syslog" in 14.04.

[Jeff Webb (jeff-webb-g)]

Thanks for being patient with me and sorry about confusing those files. I see the "start on (filesystem and started rsyslog)" now.

It seems that upstart is reporting rsyslog as started when rsyslog has been removed but not purged.

Is there an actual dependency on rsyslog (or any syslogger for that matter) in cloud-init? Cloud-config seems to run (perhaps happenstantially) without rsyslog.

[Scott Moser (smoser)]

On Tue, 8 Oct 2013, Jeff Webb wrote:

> Thanks for being patient with me and sorry about confusing those files.
> I see the "start on (filesystem and started rsyslog)" now.
>
> It seems that upstart is reporting rsyslog as started when rsyslog has
> been removed but not purged.

That seems like a bug to me. Its reporting 'started' when the start
exited non-zero. This is possibly fixed in later upstart.

> Is there an actual dependency on rsyslog (or any syslogger for that
> matter) in cloud-init? Cloud-config seems to run (perhaps
> happenstantially) without rsyslog.

It does fall back, but the provided defualt 05_logging.conf tries to use
syslog logging. And if that is not available, fall back to logging to
/var/log/cloud-init.log directly.

The reason we wait for rsyslog is that if it is not up, then writing there
will fail. I think while it was desired to succeed.

There is also some code which changes the permissions on
(cloudinit/stages.py) on the log file so that it is writable by syslog (as
it's written to/ created by cloud-init as root initially, if we didn't
change it then syslog would fail to write there).

This is probably fixable, but its not high priority to me.

well tested patches are welcome.

[Jeff Webb (jeff-webb-g)]

I think I'm finally getting the hint ;-)

I wouldn't mind taking a crack at a patch, but I'm not sure I understand what needs to be fixed yet. It sounds like we could remove the "started rsyslog" condition and it would work. Is that the right path to start heading down, or were you thinking something else? I don't think I understand why the fall back logging will fail if rsyslog didn't start. That's also not what I've observed.

Any pointers / docs for submitting a patch?

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2398