Cannot control squid "Unknown instance"

[mehturt]

Binary package hint: upstart

I cannot control squid via upstart.
Squid is running, I want to stop it:
$ sudo stop squid
stop: Unknown instance:
$ ps -ef|grep squid
proxy 1986 1 0 19:44 ? 00:00:00 (squid)
liptak 3328 1689 0 19:54 pts/1 00:00:00 grep squid
$ initctl list|grep squid
squid stop/waiting
$ sudo start squid
start: Job failed to start
$ sudo reload squid
reload: Unknown instance:

The squid is running and fully functioning, I am submitting this bug report via it.

$ lsb_release -rd
Description: Ubuntu 10.04 LTS
Release: 10.04

Using upstart 0.6.5-6 on Ubuntu 10.04 x86_64.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: upstart 0.6.5-6
ProcVersionSignature: Ubuntu 2.6.32-22.33-generic 2.6.32.11+drm33.2
Uname: Linux 2.6.32-22-generic x86_64
Architecture: amd64
Date: Sun May 2 19:50:18 2010
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: upstart

== STABLE RELEASE UPDATE ==

=== IMPACT ===

This bug affects the normal operation of squid via the reload command.

=== DEV FIX ===

Fixed in natty in the upstart job file by no longer following forks, and disabling the 'daemonizing' code in squid by passing -N. The code path that this takes has been reviewed and doesn't change the behavior of the daemon significantly.

=== TEST CASE: ===

# install squid
# verify it is running with 'sudo status squid'
# reload its configuration with 'sudo reload squid'
# verify that it is still tracked in upstart with 'sudo status squid'

The bug exists if squid is no longer visible to upstart after the 'reload' command. It is fixed if upstart still sees it running, with the same pid.

=== REGRESSION POTENTIAL ===

This does lengthen the race condition between the squid job's started event and when squid is actually listening for incoming connections. However, there are no blocking calls involved between starting squid and its daemon code forking, so the extra length is negligible. Also while the code has been reviwed and squid has been tested minimally for residual effects of the -N flag, there may be some that we did not foresee.

[Scott James Remnant (Canonical) (canonical-scott)]

This will be an issue with the squid config

[Marcus Asshauer (mcas)]

Is this a fresh install or an upgrade? If it is an upgrade do you use the old config file or the one coming with squid?

[mehturt (mehturt)]

It is an upgrade. I am using the old config file.

[mehturt (mehturt)]

I just tried with the stock config. It seems the problem appears after "reload squid" is issued.

# !ps
ps -ef|grep squid
root 4828 1899 0 17:27 pts/7 00:00:00 grep squid
# !start
start squid
squid start/running, process 4849
# !ps
ps -ef|grep squid
root 4849 1 0 17:28 ? 00:00:00 /usr/sbin/squid
proxy 4851 4849 7 17:28 ? 00:00:00 (squid)
root 4854 1899 0 17:28 pts/7 00:00:00 grep squid
# reload squid
# !ps
ps -ef|grep squid
proxy 4851 1 0 17:28 ? 00:00:00 (squid)
root 4883 1899 0 17:28 pts/7 00:00:00 grep squid
# stop squid
stop: Unknown instance:

[Vigneswaran R (r-vigneswaran)]

I did a fresh installation of Lucid. I was also not able to control squid initctl, immediately after reload.
Then, I did a "squid -k shutdown", and after that I am able to control through "initctl".

[enrigp (enrgarc)]

I have the same problem with network-manager, I cant stop, start o restart it

[ehu4ever (ehuhome)]

I'm using Squid 3.1 deb package building from source code.
It's working.

On Wed, May 12, 2010 at 8:21 PM, enrigp <email address hidden> wrote:

> I have the same problem with network-manager, I cant stop, start o
> restart it
>
> --
> Cannot control squid "Unknown instance"
> https://bugs.launchpad.net/bugs/573853
> You received this bug notification because you are subscribed to squid
> in ubuntu.
>
> Status in “squid” package in Ubuntu: New
>
> Bug description:
> Binary package hint: upstart
>
> I cannot control squid via upstart.
> Squid is running, I want to stop it:
> $ sudo stop squid
> stop: Unknown instance:
> $ ps -ef|grep squid
> proxy 1986 1 0 19:44 ? 00:00:00 (squid)
> liptak 3328 1689 0 19:54 pts/1 00:00:00 grep squid
> $ initctl list|grep squid
> squid stop/waiting
> $ sudo start squid
> start: Job failed to start
> $ sudo reload squid
> reload: Unknown instance:
>
> The squid is running and fully functioning, I am submitting this bug report
> via it.
>
> $ lsb_release -rd
> Description: Ubuntu 10.04 LTS
> Release: 10.04
>
> Using upstart 0.6.5-6 on Ubuntu 10.04 x86_64.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 10.04
> Package: upstart 0.6.5-6
> ProcVersionSignature: Ubuntu 2.6.32-22.33-generic 2.6.32.11+drm33.2
> Uname: Linux 2.6.32-22-generic x86_64
> Architecture: amd64
> Date: Sun May 2 19:50:18 2010
> ProcEnviron:
> PATH=(custom, user)
> LANG=en_US.UTF-8
> SHELL=/bin/bash
> SourcePackage: upstart
>
>
>

--
….oooO…………..
…..(….)…Oooo…
……)../…..(….)….
…..(_/…….)../…..
……………(_/……
… 路途……………
………. 遥远 …….

[Ismael Ignacio Saavedra Middleton (isma-14-94)]

error al iniciar o reiniciar squid en ubuntu migrado desde 9.04 a 10.4 lucid al reiniciar/iniciar
 /etc/init.d/squid stard

da el siguiente error

Rather than invoking init scripts through /etc/init.d, use the service(8)
utility, e.g. service squid stard

The script you are attempting to invoke has been converted to an Upstart
job, but stard is not supported for Upstart jobs.

[Karthick Murugadhas (karthickmurugadhas)]

I get the same error,

karthick@Ubuntu-desktop:~$ sudo start squid
start: Job failed to start
karthick@Ubuntu-desktop:~$ sudo stop squid
stop: Unknown instance:
karthick@Ubuntu-desktop:~$ sudo reload squid
reload: Unknown instance:
karthick@Ubuntu-desktop:~$ sudo restart squid
restart: Unknown instance:

[Hosed (liveonaware)]

I get the same error.

home@home-server:~$ sudo /etc/init.d/squid restart
Rather than invoking init scripts through /etc/init.d, use the service(8)
utility, e.g. service squid restart

Since the script you are attempting to invoke has been converted to an
Upstart job, you may also use the restart(8) utility, e.g. restart squid
squid start/running, process 4140
home@home-server:~$ sudo service squid restart
restart: Job failed to restart
home@home-server:~$

Using Ubuntu 32 bit Lucid Lynx and SQUID 2.7.STABLE7

[Eric Laganowski (eric-laganowski)]

Same issue. Pretty much stock /etc/squid/squid.conf

# dpkg -l squid
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============================-==============================-=====================================
ii squid 2.7.STABLE7-1ubuntu12 Internet object cache (WWW proxy cache)

# start -v squid
start: Job failed to start

[Peter (pry)]

This bug has a strange behaviour: Squid restarts just fine if I do it right after reboot for some time. Can't add more so far, will try to inspect logs deeper.

[FrankZequim (frankz)]

Same issue here:

# service squid reload
reload: Unknown instance:

# reload squid
reload: Unknown instance:

Only with:
# squid -k reconfigure

[goraxe (goraxe)]

the problem is squid waits for 30 (default) seconds to terminate, and prevents new squid process from spawning. I have just added this to my /etc/init/squid.conf

this causes upstart to wait for squid to fully exit before declaring it stopped, this means things like restart squid function as expected

post-stop script
        pidof \(squid\) >> /dev/null
        while [ $? -eq 0 ]; do
                sleep 1
                pidof \(squid\) || exit 0 >> /dev/null
        done
end script

The wait time for squid shutdown can be controlled with the 'shutdown_lifetime' variable
http://www.squid-cache.org/Versions/v2/2.7/cfgman/shutdown_lifetime.html

[Dave Walker (davewalker)]

@goraxe, Your change certainly looks functional, but I have concerns about the potential of waiting forever for it to terminate. I think it would be really useful if a member of the Ubuntu foundations team looks further at this. Thanks.

[Robert Schneider (robert-schneider)]

I have the very same behaviour. Using the modified post-stop script cures it a bit as it makes stop and start working again. However, reloading does NOT work.
It seems, squid terminates somehow when a -HUP is received.

[Nicolas Jungers (unbug)]

I solved it for my case by reducing the "shutdown_lifetime" to 1 seconds in "/etc/squid/squid.conf" and modifying the
script in "/etc/resolvconf/update-libc.d/squid" to replace

restart squid
by
stop squid;sleep 1;start squid

Which defeat the "restart" intended behavior but doesn't need to wait forever. Could upstart directly get the value of "shutdown_lifetime" from the squid configuration file?

[uzbu (bbubu867)]

A '/usr/bin/squid -k shutdown' returned some errors in my 'squid.conf', which I corrected.
Afterwards 'start|restart|stop squid' worked.
So it seems that '/usr/bin/squid' causes squid to run even with a faulty 'squid.conf', with warnings, but a 'start squid' does not.

[uzbu (bbubu867)]

btw, in maverick the linked '/etc/init.d/squid' won't be created anymore -> No upstart hint.

[AlainKnaff (kubuntu-misc)]

It seems that what is happening is that squid forks twice to demonize, but the "middle" process "forgets" to exit, and keeps hanging around.

# ps auxww | fgrep squid
root 2593 0.0 0.1 31792 4884 ? S 01:27 0:03 /usr/bin/perl /usr/sbin/squid-prefetch
root 14719 0.0 0.0 24796 1076 ? Ss 14:26 0:00 /usr/sbin/squid
proxy 14721 2.2 0.2 40212 10352 ? Sl 14:26 0:00 (squid)
root 14753 0.0 0.0 7576 820 pts/0 S+ 14:26 0:00 fgrep squid

# cat /var/run/squid.pid
14721

Here, 14719 is the son of the process started by init. It forks yet again, giving 14721, the program which does the actual work. Usually daemons do this in order to become fully independent of the program which launched them (so that they become inherited by init rather than whatever script launched them). However, when doing this, the middle process (14719) must exit, rather than staying around.

The result of this is that upstart becomes confused and thinks 14719 is the actual daemon, and sends it the signals instead of 14721. However, 14719 is not prepared to handle any signal, including SIGHUP, and so it exits without passing anything on to its child. Upstart notices 14719's exit, and thinks that squid is now fully gone, but it is not, as 14721 still happily chuggs along.

Adding SQUID_ARGS=-N to /etc/default/squid, and commenting out expect fork from /etc/init/squid fixes the issue.

[AlainKnaff (kubuntu-misc)]

Not adding -N, but instead replacing expect fork with expect daemon does not work as upstart will wait for the middle process to exit, which it never does.

[Amos Jeffries (yadi)]

There seems to be some confusion over the way these squid processes operates.

In your trace:
"
root 14719 0.0 0.0 24796 1076 ? Ss 14:26 0:00 /usr/sbin/squid
proxy 14721 2.2 0.2 40212 10352 ? Sl 14:26 0:00 (squid)
"

the process "/usr/sbin/squid" is a daemon supervisor. Ensuring worker "(squid)" process remains running and restarting it on failures.

Adding upstart to manage this manager process is redundant. Using -N is the correct way to start squid up to version 3.1 without the supervisor/master process. 3.2+ with multiple workers will require changes in either upstart or squid to integrate them better, but the -N mode is still available meanwhile to disable the SMP support.

[Clint Byrum (clint-fewbar)]

Amos, if I understand you right, there's no way to properly disable daemonization in squid 3.2+?

Seems that any program that forks and exits its main process on SIGHUP is unsuitable for use with 'expect fork' or 'expect daemon', so this begs the question, why can't we disable daemonization without disabling all SMP?

Anyway, that is a problem for 'squid3', if/when it gets converted to upstart. I like Alain's idea for using -N to solve squid 2.x's issue now.

[Clint Byrum (clint-fewbar)]

Setting importance to Medium as the reload command is fairly common and may disrupt management of squid.

[Robert Collins (lifeless)]

Clint, in 3.2 we use a process farm:
executed-squid (daemonises to create:
  supervisor (listens)
   worker
   worker
   worker

-N will simply mean that the executed process is the supervisor rather than a daemoniser. Details may vary but I know of no reason that that wouldn't be doable, easily, in the 3.2 main.cc.

[Launchpad Janitor (janitor)]

This bug was fixed in the package squid - 2.7.STABLE9-2.1ubuntu2

---------------
squid (2.7.STABLE9-2.1ubuntu2) natty; urgency=low

  * Run squid with -N and drop 'expect fork' to prevent HUP's from
    causing upstart to lose track of squid. (LP: #573853)
 -- Clint Byrum <email address hidden> Tue, 11 Jan 2011 10:49:37 -0800

[Clint Byrum (clint-fewbar)]

This also needs to be fixed in squid-deb-proxy, and in maverick and lucid.

I've submitted merge proposals for the lucid SRU's and for fixing squid-deb-proxy in natty.

[Launchpad Janitor (janitor)]

This bug was fixed in the package squid-deb-proxy - 0.3.4

---------------
squid-deb-proxy (0.3.4) natty; urgency=low

  * Removing expect fork and passing -N to squid to prevent upstart
    losing track of squid on reload (LP: #573853)
 -- Clint Byrum <email address hidden> Tue, 11 Jan 2011 15:07:37 -0800

[Martin Pitt (pitti)]

Accepted squid into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[mehturt (mehturt)]

I'm on maverick now, is it possible to create a package for maverick?

[Imre Gergely (cemc)]

Confirmed working on Lucid.

root@utest-lls32:~# start squid
squid start/running, process 1724

root@utest-lls32:~# status squid
squid start/running, process 1724

root@utest-lls32:~# reload squid

root@utest-lls32:~# status squid
squid start/running, process 1724

root@utest-lls32:~# ps ax |grep squid
 1724 ? Ss 0:00 /usr/sbin/squid -N
 1731 pts/0 S+ 0:00 grep --color=auto squid

root@utest-lls32:~# apt-cache policy squid
squid:
  Installed: 2.7.STABLE7-1ubuntu12.1
  Candidate: 2.7.STABLE7-1ubuntu12.1
  Version table:
 *** 2.7.STABLE7-1ubuntu12.1 0
        500 http://ro.archive.ubuntu.com/ubuntu/ lucid-proposed/main Packages
        100 /var/lib/dpkg/status
     2.7.STABLE7-1ubuntu12 0
        500 http://ro.archive.ubuntu.com/ubuntu/ lucid/main Packages

[Martin Senft (martinsenft)]

Works when installed on Maverick.

root@localhost:~# status squid
squid start/running, process 13708

root@localhost:~# reload squid

root@localhost:~# status squid
squid start/running, process 13708

root@localhost:~# restart squid
squid start/running, process 14256

root@localhost:~# status squid
squid start/running, process 14256

root@localhost:~# ps -fC squid
UID PID PPID C STIME TTY TIME CMD
proxy 14256 1 0 14:39 ? 00:00:00 /usr/sbin/squid -N

root@localhost:~# apt-cache policy squid
squid:
  Installed: 2.7.STABLE7-1ubuntu12.1
  Candidate: 2.7.STABLE9-2ubuntu5
  Version table:
     2.7.STABLE9-2ubuntu5 0
        990 http://cz.archive.ubuntu.com/ubuntu/ maverick/main amd64 Packages
 *** 2.7.STABLE7-1ubuntu12.1 0
        500 http://cz.archive.ubuntu.com/ubuntu/ lucid-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

[Imre Gergely (cemc)]

I don't think you should install the Lucid package on Maverick though. When there will be a package in maverick-proposed, you can test that one.

[Launchpad Janitor (janitor)]

This bug was fixed in the package squid - 2.7.STABLE7-1ubuntu12.1

---------------
squid (2.7.STABLE7-1ubuntu12.1) lucid-proposed; urgency=low

  * Run squid with -N and drop 'expect fork' to prevent HUP's from
    causing upstart to lose track of squid. (LP: #573853)
 -- Clint Byrum <email address hidden> Tue, 01 Feb 2011 14:26:33 +0100

[Pål Andreassen (paal-andreassen)]

I'm not quite into this package branching thing. When will this fix be released in the lucid/main repository?

[Clint Byrum (clint-fewbar)]

On Tue, 2011-03-01 at 14:06 +0000, Pananza wrote:
> I'm not quite into this package branching thing. When will this fix be
> released in the lucid/main repository?
>

Pananza, the package that fixes this bug has been released into
lucid-updates already.

[Martin Senft (martinsenft)]

I believe that the patch made for Natty should work on Maverick as well. Would anyone more experienced care to make the Maverick package, please? Clint?

Current Maverick-amd64:

------------------------------------------------------------
root@localhost:~# apt-cache policy squid
squid:
  Installed: 2.7.STABLE9-2ubuntu5
  Candidate: 2.7.STABLE9-2ubuntu5
  Version table:
 *** 2.7.STABLE9-2ubuntu5 0
        990 http://ftp.cvut.cz/ubuntu/ maverick/main amd64 Packages
        100 /var/lib/dpkg/status

root@localhost:~# start squid
squid start/running, process 14874

root@localhost:~# status squid
squid start/running, process 14874

root@localhost:~# reload squid

root@localhost:~# status squid
squid stop/waiting
------------------------------------------------------------

Current Maverick-amd64 using /etc/init/squid.conf from Natty package squid_2.7.STABLE9-2.1ubuntu3_amd64.deb:

------------------------------------------------------------
root@localhost:~# diff /etc/init/squid.conf squid.conf.natty
11,12d10
< expect fork
<
81c79
< exec /usr/sbin/squid $SQUID_ARGS
---
> exec /usr/sbin/squid -N $SQUID_ARGS

root@localhost:~# cp squid.conf.natty /etc/init/squid.conf

root@localhost:~# killall squid

root@localhost:~# start squid
squid start/running, process 15035

root@localhost:~# status squid
squid start/running, process 15035

root@localhost:~# reload squid

root@localhost:~# status squid
squid start/running, process 15035

root@localhost:~# restart squid
squid start/running, process 15122

root@localhost:~# status squid
squid start/running, process 15122
------------------------------------------------------------

[Thomas (t.c)]

I also have the problem after reload squid service:

apt-cache policy squid
squid:
  Installiert: 2.7.STABLE7-1ubuntu12
  Kandidat: 2.7.STABLE7-1ubuntu12.2
  Versions-Tabelle:
     2.7.STABLE7-1ubuntu12.2 0
        500 http://de.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
 *** 2.7.STABLE7-1ubuntu12 0
        500 http://de.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status

pls make a update for lucid too! It is really important issue!!

The fix from Comment 38 also works in lucid

[Martin Senft (martinsenft)]

Thomas C., the fix for Lucid was already released. You just need to update to version 2.7.STABLE7-1ubuntu12.2 from your current version 2.7.STABLE7-1ubuntu12 as suggested by apt (see Installiert and Kandidat lines).

[Mahyuddin Susanto (udienz)]

Thomas, as martin sya please upgrade to latest release. i use squid 2.7.STABLE7-1ubuntu12.2, and i can confirm that this problem is solved. squid running very smooth. Thanks Clint!

[Alecz20 (alexguzu)]

Is the updated package in the main or still in proposed?

I have run apt-get update and apt-get dist-upgrade, but nothing really happened.
It says that there are no packages to update.

[Imre Gergely (cemc)]

IF you're using Lucid it's there in updates

gimre@voy:~$ apt-cache policy squid
squid:
  Installed: 2.7.STABLE7-1ubuntu12.3
  Candidate: 2.7.STABLE7-1ubuntu12.3
  Version table:
 *** 2.7.STABLE7-1ubuntu12.3 0
        500 http://ro.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        100 /var/lib/dpkg/status
     2.7.STABLE7-1ubuntu12 0
        500 http://ro.archive.ubuntu.com/ubuntu/ lucid/main Packages

I've tested it and it seems to work fine.

[Skilly (michael-scheepers)]

It seems this bug is back with squid version 2.7.STABLE7-1ubuntu12.4 on Lucid ...

[Clint Byrum (clint-fewbar)]

Hi Skilly, it doesn't make sense that this would cause the bug to regress:

http://launchpadlibrarian.net/78309117/squid_2.7.STABLE7-1ubuntu12.3_2.7.STABLE7-1ubuntu12.4.diff.gz

It didn't touch the default args of squid at all. Make sure you don't have a modified /etc/init/squid.conf, the exec line should look like this:

 exec /usr/sbin/squid -N $SQUID_ARGS

[Skilly (michael-scheepers)]

On 22 November 2011 05:44, Clint Byrum <email address hidden> wrote:

> Hi Skilly, it doesn't make sense that this would cause the bug to
> regress:
>
>
> http://launchpadlibrarian.net/78309117/squid_2.7.STABLE7-1ubuntu12.3_2.7.STABLE7-1ubuntu12.4.diff.gz
>
> It didn't touch the default args of squid at all. Make sure you don't
> have a modified /etc/init/squid.conf, the exec line should look like
> this:
>
> exec /usr/sbin/squid -N $SQUID_ARGS
>
> Thanks for your reply Clint. Based to your info provided above, I got a
more meaningful error message and discovered that the problem was due to
permissions set on /var/log/squid.

For some reason permission was set to 664 (ownership of root:proxy) and the
app requires the proxy user to have WRITE access to the squid folder. I
changes the permissions to 774 and this resolved the problem.

Not sure how the permissions issue came into being as I had been running
squid successfully for a number of months, before the issue occurred.

My apologies for reporting this as a bug, I'm probably messed up the
permissions while playing on the server.

Cheers,

Michael

[Clint Byrum (clint-fewbar)]

Excerpts from Skilly's message of Wed Nov 23 12:01:39 UTC 2011:
> On 22 November 2011 05:44, Clint Byrum <email address hidden> wrote:
>
> > Hi Skilly, it doesn't make sense that this would cause the bug to
> > regress:
> >
> >
> > http://launchpadlibrarian.net/78309117/squid_2.7.STABLE7-1ubuntu12.3_2.7.STABLE7-1ubuntu12.4.diff.gz
> >
> > It didn't touch the default args of squid at all. Make sure you don't
> > have a modified /etc/init/squid.conf, the exec line should look like
> > this:
> >
> > exec /usr/sbin/squid -N $SQUID_ARGS
> >
> > Thanks for your reply Clint. Based to your info provided above, I got a
> more meaningful error message and discovered that the problem was due to
> permissions set on /var/log/squid.
>
> For some reason permission was set to 664 (ownership of root:proxy) and the
> app requires the proxy user to have WRITE access to the squid folder. I
> changes the permissions to 774 and this resolved the problem.
>
> Not sure how the permissions issue came into being as I had been running
> squid successfully for a number of months, before the issue occurred.
>
> My apologies for reporting this as a bug, I'm probably messed up the
> permissions while playing on the server.

No worries! Thanks for taking the time to file the report! Don't hesitate
to file more in the future!

[Rolf Leggewie (r0lf)]

maverick has seen the end of its life and is no longer receiving any updates. Marking the maverick task for this ticket as "Won't Fix".

[Rolf Leggewie (r0lf)]

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".