seamless reload disconnects clients poorly
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
New
|
Undecided
|
Unassigned |
Bug Description
I'm seeing some ungraceful disconnects with seamless-reload on the proxy:
Traceback (most recent call last):
File "/usr/local/
func(*args, **kwargs)
File "/vagrant/
container_name, name, http_conn=conn)
File "/vagrant/
conn.
File "/vagrant/
files=files, **self.
File "/vagrant/
return self.request_
File "/usr/local/
resp = self.send(prep, **send_kwargs)
File "/usr/local/
r = adapter.
File "/usr/local/
raise ConnectionError
ConnectionError: ('Connection aborted.', BadStatusLine('No status line received - the server has closed the connection',))
I'm reloading like:
vagrant@saio:~$ swift-init reload-seamless proxy
Signal proxy-server pid: 9172 signal: 10
Signal proxy-server pid: 9173 signal: 10
It's not exactly clear to me what the expected behavior is during a proxy-server reload-seamless for existing connections that are presumably trying to make pipeline requests. I suppose ideally the server would send Connection: close in the response. But if the client keeps sending requests at somepoint I suppose it would close the connection - so it's possible this is a client side issue and swift-bench bug?
I think this is going to be unavoidable -- it'd be *nice* if we could get a `Connection: close` header out, and we might even be able to do that for writes or even account/container reads. But any connections where we've already sent headers when the SIGUSR1 was received are out of luck -- the server's hanging up and there's no good way to signal that to the client.