civetweb: 0x7f4763896580: cannot bind to 70: 13 (Permission denied)

[Frode Nordahl]

Something changed between ceph 15.1.x and ceph 15.2.x which prohibits civetweb from binding to ports < 1024.

The packages have been running ceph radosgw as a non-root user since forever, so I'm not quite sure what changed where, so this is mainly a bug to track what may very well be a upstream issue.

[James Page (james-page)]

I'm not sure the radosgw processes ran as the ceph user - I'd need to check

That might be the change that has caused this issue.

[James Page (james-page)]

Nope - ran as ceph/ceph at eoan as well...

[James Page (james-page)]

2020-04-08T08:06:28.586+0000 7fd9b440d980 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-04-08T08:06:28.586+0000 7fd9b440d980 0 ceph version 15.2.0 (dc6a0b5c3cbf6a5e1d6d4f20b5ad466d76b96247) octopus (rc), process radosgw, pid 22337
2020-04-08T08:06:28.586+0000 7fd9b440d980 0 framework: civetweb
2020-04-08T08:06:28.586+0000 7fd9b440d980 0 framework conf key: port, val: 70
2020-04-08T08:06:28.586+0000 7fd9b440d980 1 radosgw_Main not setting numa affinity
2020-04-08T08:06:28.686+0000 7fd9b440d980 0 framework: beast
2020-04-08T08:06:28.686+0000 7fd9b440d980 0 framework conf key: ssl_certificate, val: config://rgw/cert/$realm/$zone.crt
2020-04-08T08:06:28.686+0000 7fd9b440d980 0 framework conf key: ssl_private_key, val: config://rgw/cert/$realm/$zone.key
2020-04-08T08:06:28.686+0000 7fd9b440d980 0 starting handler: civetweb
2020-04-08T08:06:28.686+0000 7fd9b440d980 0 civetweb: 0x7fd9b4203580: cannot bind to 70: 13 (Permission denied)

[James Page (james-page)]

https://github.com/ceph/ceph/commit/e28718eaa18e49c770db45820b591088ea92846b moves the creation of the global ceph context to before the determination of the frontend for the rgw so the flag that defers the privileged drop in the context setup is not set at the right point in time:

https://github.com/ceph/ceph/blob/master/src/rgw/rgw_main.cc#L234

so privs are dropped before the frontend has been determined.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ceph - 15.2.0-0ubuntu2

---------------
ceph (15.2.0-0ubuntu2) focal; urgency=high

  * d/p/revert-rgw-move-frontends-initial-init-to-after-glob.patch:
    Revert change to initialize global ceph context before determination
    of the frontend in use, ensuring that privs are not dropped before
    any frontend port binding to ports < 1024 has been completed
    (LP: #1869324).

 -- James Page <email address hidden> Wed, 08 Apr 2020 10:39:39 +0100