HAProxy fails to start at boot time

[Simon Bazley]

Binary package hint: haproxy

I've got haproxy installed and configured on one of my servers. In the first 'listen' section of the config file, it binds to a number of aliased IP addresses. While I can start the service fine from the login prompt, it would not start at boot time, instead it showed the following in /var/log/boot.log

 * Starting haproxy haproxy [ALERT] 346/160552 (927) : Starting proxy haproxy: cannot bind socket
[ALERT] 346/160552 (927) : Starting proxy haproxy: cannot bind socket
[ALERT] 346/160552 (927) : Starting proxy haproxy: cannot bind socket
[ALERT] 346/160552 (927) : Starting proxy haproxy: cannot bind socket
[ALERT] 346/160552 (927) : Starting proxy haproxy: cannot bind socket
[ALERT] 346/160552 (927) : Starting proxy haproxy: cannot bind socket
[ALERT] 346/160552 (927) : Starting proxy haproxy: cannot bind socket
                                                                         [fail]

On inspection the problem seems to come down to the lack of and specified start/stop numbers in the sysV init scripts, so it defaults to S20/K20 for everything. While the correct solution is to write an upstart script for haproxy, in the mean time, it should be set, so that the start number is higher than 35 (which seems to be the number /etc/init.d/networking uses)/

Changing the startup number to 35 seems to fix it, but I think 36 would be safer (the old number for networking was 35, so best make it start after that).

This fixed the problem, on my server:-

update-rc.d -f haproxy remove
update-rc.d haproxy start 36 2 3 4 5 . stop 20 0 1 6 .

[Simon Bazley (sibaz)]

Just a quick follow up, that didn't fix it. It went away for a bit, then magically came back. The solution seemed to be to add networking to init.d startup directories before haproxy:-

update-rc.d networking start 35 2 3 4 5 .

Like I said, the solution is probably to create a proper upstart script

[Paul Sladen (sladen)]

Is this specific bug still happening with the Ubuntu 11.04 beta?

(However, not that /restarting/ of haproxy may be broken for other reasons at bug #731878 "shutdown(2) behavior changed in kernel").

[Steve Kamerman (stevekamerman)]

There was a fairly significant bugfix in HAProxy v 1.4.15, April 2011 that prevents DOS attacks and HAProxy crashes. I've built a .deb for it with the same config options as the existing version (TCP Splicing, Full Transparent Proxy, PCRE). Networking is listed as a startup dependency, so it should fall after that in the startup order. Can you test it to make sure it's starting up properly?

http://www.stevekamerman.com/haproxy/

I've got to translate the changelog from the haproxy site into deb format yet, but I'll submit the package to Ubuntu after that.

[Scott Kitterman (kitterman)]

haproxy was updated in Debian, but not Ubuntu. I've requested to have the package sync'ed so we'll have the newest for 11.10.

[Scott Kitterman (kitterman)]

Ubuntu 11.10 was released with haproxy 1.4.15.

[Scott Kitterman (kitterman)]

From 10.10 (Maverick) on, the haproxy init has:

# Required-Start: $local_fs $network $remote_fs

This looks right to me. In 10.04 (Lucid) it has only:

# Required-Start: $local_fs $network

I suspect this is what your bug was and it's fixed. Did you see this problem on 10.04 originally and have you seen it on a later version since?

[Simon Bazley (sibaz)]

I've not tried this on a newer version of ubuntu, as I only tend to use
LTS versions. I haven't got my head around the new post sys V startup
thing, but the issue was that it needs to start after networking, but
was trying to start before it.

I would assume from what you're saying, that in 10.04 it was configured
to 'require' local_fs and network to have started. That would make
sense, but it didn't seem to be doing that in my case, as it was trying
to start before the aliased IPs had started. Not sure why they wouldn't
have been included in the 'network' command.

Also not sure that adding 'remote_fs' would help, unless that somehow
adds the dependancy for the aliased IPs, when networking doesn't. I'll
check this out when I next have time to try out a new desktop OS.

Thanks Scott,

Simon

On 21/12/11 20:49, Scott Kitterman wrote:
> > From 10.10 (Maverick) on, the haproxy init has:
>
> # Required-Start: $local_fs $network $remote_fs
>
> This looks right to me. In 10.04 (Lucid) it has only:
>
> # Required-Start: $local_fs $network
>
> I suspect this is what your bug was and it's fixed. Did you see this
> problem on 10.04 originally and have you seen it on a later version
> since?
>
>
> ** Changed in: haproxy (Ubuntu)
> Status: New => Incomplete
>

[Launchpad Janitor (janitor)]

[Expired for haproxy (Ubuntu) because there has been no activity for 60 days.]