Activity log for bug #1751842

Date Who What changed Old value New value Message
2018-02-26 17:17:35 Jason Hobbs bug added bug
2018-02-26 17:17:35 Jason Hobbs attachment added juju-crashdump-fd59bef7-c249-4b87-a342-53ea17ececba.tar.gz https://bugs.launchpad.net/bugs/1751842/+attachment/5063613/+files/juju-crashdump-fd59bef7-c249-4b87-a342-53ea17ececba.tar.gz
2018-02-26 17:22:40 Jason Hobbs description We have hit several failures of this pattern. This is using HA glance with ceph as the back end. Sometimes this occurs right after we finish the openstack deploy and are uploading images: http://paste.ubuntu.com/p/hm7K7FfYsB/ At least once it's happened during a rally test. Here is the bundle and overlay: http://paste.ubuntu.com/p/Rhf4FN2kJ3/ We have hit several failures of this pattern. This is using HA glance with ceph as the back end. Sometimes this occurs right after we finish the openstack deploy and are uploading images: http://paste.ubuntu.com/p/hm7K7FfYsB/ At least once it's happened during a rally test. Here is the bundle and overlay: http://paste.ubuntu.com/p/Rhf4FN2kJ3/ Despite the error message, I think it can connect, because glance-api.log has these log messages: http://paste.ubuntu.com/p/zRrR8QczXN/ 10.244.40.215 is the ip of the glance 0 unit, and the above log comes from the glance 1 unit. I think it's haproxy from glance 0 forwarding the connection to glance 1. I think this is probably a timeout problem; maybe haproxy needs to be adjusted? This is also suspicious, regarding upload speed: http://paste.ubuntu.com/p/g5f6fhyc2M/
2018-02-26 17:34:07 Jason Hobbs description We have hit several failures of this pattern. This is using HA glance with ceph as the back end. Sometimes this occurs right after we finish the openstack deploy and are uploading images: http://paste.ubuntu.com/p/hm7K7FfYsB/ At least once it's happened during a rally test. Here is the bundle and overlay: http://paste.ubuntu.com/p/Rhf4FN2kJ3/ Despite the error message, I think it can connect, because glance-api.log has these log messages: http://paste.ubuntu.com/p/zRrR8QczXN/ 10.244.40.215 is the ip of the glance 0 unit, and the above log comes from the glance 1 unit. I think it's haproxy from glance 0 forwarding the connection to glance 1. I think this is probably a timeout problem; maybe haproxy needs to be adjusted? This is also suspicious, regarding upload speed: http://paste.ubuntu.com/p/g5f6fhyc2M/ We have hit several failures of this pattern. This is using HA glance with ceph as the back end. Sometimes this occurs right after we finish the openstack deploy and are uploading images: http://paste.ubuntu.com/p/hm7K7FfYsB/ At least once it's happened during a rally test. Here is the bundle and overlay: http://paste.ubuntu.com/p/Rhf4FN2kJ3/ Despite the error message, I think it can connect, because glance-api.log has these log messages: http://paste.ubuntu.com/p/zRrR8QczXN/ 10.244.40.215 is the ip of the glance 0 unit, and the above log comes from the glance 1 unit. I think it's haproxy from glance 0 forwarding the connection to glance 1. I think this is probably a timeout problem; maybe haproxy needs to be adjusted? This is also suspicious, regarding upload speed: http://paste.ubuntu.com/p/g5f6fhyc2M/ Some example failures from rally: http://paste.ubuntu.com/p/ZPbFD4WNRZ/
2018-02-27 19:35:18 Chris Gregan bug added subscriber Canonical Field High
2018-02-27 22:01:40 Jason Hobbs attachment added juju-crashdump-c8777874-41ea-4ead-83b3-54e0793f8d18.tar.gz https://bugs.launchpad.net/charm-glance/+bug/1751842/+attachment/5064821/+files/juju-crashdump-c8777874-41ea-4ead-83b3-54e0793f8d18.tar.gz
2018-02-28 23:44:51 David Ames charm-glance: status New Confirmed
2018-02-28 23:44:58 David Ames charm-glance: importance Undecided Medium
2018-02-28 23:45:07 David Ames charm-glance: assignee David Ames (thedac)
2018-02-28 23:45:11 David Ames charm-glance: milestone 18.02
2018-03-02 00:40:16 David Ames charm-glance: status Confirmed Invalid
2018-03-02 00:40:37 David Ames bug task added charm-ceph-mon
2018-03-02 00:40:49 David Ames bug task added charm-ceph-osd
2018-03-02 16:21:25 David Ames charm-ceph-mon: status New Invalid
2018-03-02 16:21:28 David Ames charm-ceph-osd: status New Invalid
2018-03-02 16:34:29 Jason Hobbs charm-glance: status Invalid New
2018-03-02 16:34:31 Jason Hobbs charm-ceph-osd: status Invalid New
2018-03-02 16:34:33 Jason Hobbs charm-ceph-mon: status Invalid New
2018-03-02 17:09:43 Chris MacNaughton charm-ceph-mon: assignee Chris MacNaughton (chris.macnaughton)
2018-03-02 17:09:45 Chris MacNaughton charm-ceph-osd: assignee Chris MacNaughton (chris.macnaughton)
2018-03-02 17:10:51 David Ames charm-glance: status New Invalid
2018-03-15 12:31:27 James Page charm-ceph-osd: status New Incomplete
2018-03-15 12:31:29 James Page charm-ceph-mon: status New Incomplete
2018-03-19 15:10:31 Jason Hobbs charm-ceph-mon: status Incomplete New
2018-03-19 15:10:33 Jason Hobbs charm-ceph-osd: status Incomplete New
2018-03-22 09:09:45 James Page charm-ceph-mon: status New Incomplete
2018-03-22 09:09:47 James Page charm-ceph-osd: status New Incomplete
2018-04-19 22:08:46 Chris Gregan removed subscriber Canonical Field High
2019-01-31 16:14:56 Chris Gregan attachment added juju-crashdump-openstack-2019-01-29-21.15.17.tar.gz https://bugs.launchpad.net/charm-glance/+bug/1751842/+attachment/5234792/+files/juju-crashdump-openstack-2019-01-29-21.15.17.tar.gz