Nginx 1.8 (stable PPA) service commands don't work.

[Lior Goikhburg]

After upgrade of official trusty Nginx package to 1.8 from the stable repo, I'm having a problem with 'service nginx reload' command:

root@bal01 ~ # service nginx reload
reload: Job is not running: nginx
root@bal01 ~ #

root@bal01 ~ # service nginx stop
root@bal01 ~ #
But the processes are not stopped

Running /etc/init.d/nginx reload or /etc/init.d/nginx stop works

[Thomas Ward (teward)]

I've seen this as an issue myself. The workaround: terminate the current nginx processes with `sudo killall nginx`, then do `sudo service nginx start`.

I'm not sure why it's failing to detect the older instances. If it happens again after that command/workaround let me know.

[Lior Goikhburg (goikhburg)]

Workaround:
removed /etc/init/nginx.conf to rely on the init.d scripts instead.
All service commands work now.

BTW: /etc/init/nginx.conf is not present in the official trusty nginx package.

[Lior Goikhburg (goikhburg)]

weird!
This is what happens when /etc/init/nginx.conf exists:

root@bal01 ~ # killall nginx; sudo service nginx start
nginx: no process found
start: Job is already running: nginx
root@bal01 ~ # service nginx status
nginx start/running
root@bal01 ~ # /etc/init.d/nginx status
 * nginx is not running
root@bal01 ~ # service nginx start
start: Job is already running: nginx
root@bal01 ~ #

[Lior Goikhburg (goikhburg)]

This all happens because both initialization configs (upstart: /etc/init/nginx and RC: /etc/init.d/nginx) are present.
The presence of both approaches creates a conflict.

man service:
The existence of an upstart job of the same name as a script in /etc/init.d will cause the upstart job to take precedence over the init.d script.

All extra commands (such as: configtest, rotate, upgrade) that are present in /etc/init.d/nginx script will not work with 'service' until nginx upstart config is removed.

As for the reload command: If nginx was started via /etc/init.d/nginx start and nginx upstart config is present, the command will not work as the daemon is not being controlled by upstart.

IMHO: nginx package should be built without upstart support (like the official trusy package)

[Lior Goikhburg (goikhburg)]

For my systems (in the SCM) I've performed the following as a workaround:

1 stop nginx (to get it out of upstart control): # initctl stop nginx
2 remove nginx upstart config /etc/init/nginx: # rm /etc/init/nginx
3 start nginx (via RC script): # service nginx start
4 configure nginx to run at startup: # update-rc.d nginx defaults

[Lior Goikhburg (goikhburg)]

Related https://bugs.launchpad.net/nginx/+bug/1450770

[triplepoint (triplepoint)]

I was just bitten by this myself. For what it's worth, in Ansible, I got around it with doing this immediately after the nginx package install (this essentially Lior's solution, converted into Ansible tasks):

---
# We need to remove the Upstart config, which is conflicting with the also-included systemV init.d config
# Re: https://bugs.launchpad.net/nginx/+bug/1476296

- name: Stop nginx (as managed by upstart)
  command: initctl stop nginx

- name: Remove the upstart init command
  file: state=absent path=/etc/init/nginx.conf

- name: ensure that Nginx will start on restarts with systemV
  command: update-rc.d nginx defaults

- name: ensure that nginx is started
  service: name=nginx state=started

[Bob (v-bob)]

I also affected by the same behaviour when upgrading from stable 1.8 to mainline 1.9

[Lior Goikhburg (goikhburg)]

Nobody seems to care.
As for me, I'm waiting for the release of Xenial, which will have official supported package of Nginx 1.9.x

[Thomas Ward (teward)]

Lior: More a case of us being overworked - only a few people work on the PPA packages, and I personally have a lot of work for the Xenial prep :P I'll look into this when I can.

[Daniel van Dorp (djvdorp)]

Just like Bob (v-bob), I am also affected by the same behaviour when upgrading from stable 1.8 to mainline 1.9. Thanks for your workaround Lior Goikhburg (goikhburg), it worked like a charm, I just had to kill some lingering nginx processes by hand after your step 1 via:

# kill $(ps aux | grep '[n]ginx' | awk '{print $2}')

[Thomas Ward (teward)]

It looks like this is an issue in Trusty and possibly Precise?

I'm working on a fix currently.

[Thomas Ward (teward)]

Hello to all affected.

For Precise and Trusty, I have made a package revision that needs further testing before I push it to the main repositories. Please add the following PPA and test the latest nginx versions in that PPA (nginx 1.10.1-0+trusty1 for Trusty, and nginx 1.10.1-0+precise1 for Precise):

sudo add-apt-repository ppa:teward/nginx-temporary
sudo apt-get update
sudo apt-get upgrade

If the issue is resolved with these packages, please let me know, so I can push the update to the main repository for Stable releases. (Mainline will get the updates at a later time)

[Daniel van Dorp (djvdorp)]

Hi Thomas (teward),

Thanks for your work on the fix, and sorry for my slow reply, I needed some time to setup a test environment to reproduce this bug. Forgot to mention it earlier, but I encountered the issue on Ubuntu 12.04 LTS (Precise) which is why I have also just reproduced it in an equal environment.

Relevant steps taken to confirm the fix works on Precise:

1) Obtain a clean Ubuntu 12.04 LTS (Precise) installation
2) apt-get update && apt-get upgrade && apt-get install nginx # will give you stable nginx
3) sudo add-apt-repository ppa:teward/nginx-temporary # teward steps
4) sudo apt-get update # teward steps
5) sudo apt-get upgrade (or just apt-get install nginx) # teward steps, will give you his fixed nginx
6) add nginx.org mainline for precise, then apt-get update again, won't work right away so you need apt-get remove nginx && apt-get autoremove before installing mainline from nginx.org
7) sudo apt-get upgrade (or just apt-get install nginx) # will give you nginx.org mainline nginx

Between all changed versions of nginx (stable -> teward fixed version -> mainline nginx.org) I was able to confirm that all related service and init.d commands remained working and are correct, so I can confirm that this fix for Ubuntu 12.04 LTS (Precise) works like charm, thanks Thomas!

[Thomas Ward (teward)]

Daniel:

Thanks for testing. However, keep in mind I'm also going to apply these fixes to the Mainline branch of the PPA as well.

Given they work in Precise, I"m assuming they'll work for Trusty. Going to upload them to Stable shortly, then apply the changes to Mainline later.

[Daniel van Dorp (djvdorp)]

Hi Thomas (teward),

I just did the exact same tests in Trusty and can confirm that this fix also works like a charm on Ubuntu 14.04 LTS (Trusty), however, as Lior Goikhburg (goikhburg) wrote above, the file /etc/init.d/nginx.conf is not present in the official trusty nginx package so I was unable to reproduce the issue I had on Precise there, however as said, I can confirm that upgrading works fine and all related service and init.d commands remained working and are correct.

Thanks again!

[alican (alican)]

Hi Thomas Ward (teward)

Now there is another problem: I cannot simply run nginx restart on ubuntu 14 box.

When I do, I receive this error:

root@web:~# service nginx restart
 * Restarting nginx nginx nginx: [emerg] bind() to 0.0.0.0:8081 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:8081 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:8081 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:8081 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:8081 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
nginx: [emerg] still could not bind()

[Thomas Ward (teward)]

Alican: Different issue, different bug please.

Do `sudo killall nginx` then try and restart the service. Sounds like leftover processes. Will have to tweak PPAs again after this thunderstorm here goes away.