Possible to speedup chownr?
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ceph OSD Charm |
Fix Released
|
Undecided
|
Ponnuvel Palaniyappan | ||
Charm Helpers |
Fix Committed
|
Undecided
|
Ponnuvel Palaniyappan |
Bug Description
I wonder if `chownr` function (recursive chown) can be improved.
It calls `os.path.lexists` and `os.path.exists` for *every* file. The relevant source code from https:/
for root, dirs, files in os.walk(path, followlinks=
for name in dirs + files:
full = os.path.join(root, name)
if not broken_symlink:
I think the purpose of the extra check (not broken symlink) is to avoid calling chown on broken symlinks. Would rewriting this part to:
for root, dirs, files in os.walk(path, followlinks=
for name in dirs + files:
full = os.path.join(root, name)
try:
except FileNotFoundError:
do the equivalent functionality? This would avoid nearly 2 system calls for every file.
Ordinarily, these "extra" system calls wouldn't be an issue for a directory with few files. But in case of ceph-osd charm where there can be a massive number of files, this is noticeable (few mins to hours - depending on the number of files).
tags: | added: seg |
Changed in charm-helpers: | |
status: | In Progress → Fix Committed |
Changed in charm-ceph-osd: | |
status: | In Progress → Fix Committed |
Changed in charm-ceph-osd: | |
milestone: | none → 21.01 |
Changed in charm-ceph-osd: | |
status: | Fix Committed → Fix Released |
Github fix proposal: https:/ /github. com/juju/ charm-helpers/ pull/516