Ship arcstat.py and arc_summary.py with zfsutils-linux
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
zfs-linux (Ubuntu) |
Fix Released
|
Medium
|
Eric Desrochers | ||
Xenial |
Fix Released
|
Medium
|
Eric Desrochers | ||
Yakkety |
Fix Released
|
Medium
|
Eric Desrochers |
Bug Description
[Impact]
zfsutils-linux missing some important tools.
Python3 compatible scripts : arc_summary.py, arcstat.py and dbufstat.py found in src_code/cmd/ are not taken into account during compilation.
* arc_summary.py : Provides a summary of the statistics
https:/
* arcstat.py : Print out ZFS ARC Statistics exported via kstat(1)
https:/
* dbufstat.py : Print out statistics for all cached dmu buffers
https:/
[Test Case]
# rmadison zfsutils-linux
...
zfsutils-linux | 0.6.5.6-0ubuntu10 | xenial-updates
...
# List contents of a deb package.
$ dpkg -c zfsutils-
# You will notice there are not present.
[Regression Potential]
None expected, debian package[1] has already made the change to build the python scripts.
[1]
Package: zfsutils-linux
Source: zfs-linux
Version: 0.6.5.7-1
Architecture: amd64
Maintainer: Debian ZFS on Linux maintainers <email address hidden>
# List contents of a deb package.
$ dpkg -c zfsutils-
-rwxr-xr-x root/root 40751 2016-05-31 03:40 ./bin/arc_
-rwxr-xr-x root/root 13057 2016-05-31 03:40 ./bin/arcstat.py
-rwxr-xr-x root/root 18718 2016-05-31 03:40 ./bin/dbufstat.py
[Other Info]
* This is based on the following Debian commits :
Debian git commits
===
commit 730989397063955
Author: Aron Xu <email address hidden>
Date: Tue Apr 26 17:01:29 2016 +0800
Add dh-python to b-d
--
commit 814413118fb88de
Author: Aron Xu <email address hidden>
Date: Sun Nov 22 15:44:16 2015 +0800
Add python related dependencies
===
* Despite what Debian did by adding theses script under /bin.
I'll add them under /usr/sbin, which is more appropriate.
* Unfortunately manpage for theses 3 program seems limited.
There is none in Debian nor git upstream repo.
The only manpage I found is for arcstat online :
http://
[Original Description]
arcstat.py and arc_summary.py are valuable tools to determine ZFS' ARC usage, it is not obvious why they are not included in zfsutils-linux. As ubuntu-minimal already depends on python3 it should be safe to assume python is available, or am I mistaken here?
arcstat.py gives an iostat-like overview about ARC reads, hit rate, current and target size in regular intervals:
# ./arcstat.py 1
time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c
19:22:33 0 0 0 0 0 0 0 0 0 2.0G 7.8G
19:22:34 3 0 0 0 0 0 0 0 0 2.0G 7.8G
19:22:35 21 0 0 0 0 0 0 0 0 2.0G 7.8G
^C
arc_summary.py shows a more detailed overview of the current ARC status and ZFS tunables:
# ./arc_summary.py
-------
ZFS Subsystem Report Sun Apr 24 19:23:25 2016
ARC Summary: (HEALTHY)
Memory Throttle Count: 0
ARC Misc:
Deleted: 46
Mutex Misses: 0
Evict Skips: 0
ARC Size: 25.10% 1.95 GiB
Target Size: (Adaptive) 100.00% 7.78 GiB
Min Size (Hard Limit): 0.40% 32.00 MiB
Max Size (High Water): 248:1 7.78 GiB
ARC Size Breakdown:
Recently Used Cache Size: 50.00% 3.89 GiB
Frequently Used Cache Size: 50.00% 3.89 GiB
ARC Hash Breakdown:
Elements Max: 32.31k
Elements Current: 99.78% 32.24k
Collisions: 40.54k
Chain Max: 3
Chains: 240
ARC Total accesses: 4.54m
Cache Hit Ratio: 99.39% 4.51m
Cache Miss Ratio: 0.61% 27.74k
Actual Hit Ratio: 98.76% 4.48m
Data Demand Efficiency: 99.73% 3.23m
Data Prefetch Efficiency: 11.32% 6.41k
CACHE HITS BY CACHE LIST:
Anonymously Used: 0.64% 28.65k
Most Recently Used: 21.90% 987.29k
Most Frequently Used: 77.47% 3.49m
Most Recently Used Ghost: 0.00% 0
Most Frequently Used Ghost: 0.00% 0
CACHE HITS BY DATA TYPE:
Demand Data: 71.40% 3.22m
Prefetch Data: 0.02% 725
Demand Metadata: 27.97% 1.26m
Prefetch Metadata: 0.62% 27.92k
CACHE MISSES BY DATA TYPE:
Demand Data: 31.81% 8.82k
Prefetch Data: 20.48% 5.68k
Demand Metadata: 21.99% 6.10k
Prefetch Metadata: 25.72% 7.13k
File-Level Prefetch: (HEALTHY)
DMU Efficiency: 36.39m
Hit Ratio: 93.36% 33.97m
Miss Ratio: 6.64% 2.42m
Colinear: 2.42m
Hit Ratio: 0.02% 505
Miss Ratio: 99.98% 2.42m
Stride: 33.94m
Hit Ratio: 100.00% 33.94m
Miss Ratio: 0.00% 14
DMU Misc:
Reclaim: 2.42m
Successes: 2.46% 59.51k
Failures: 97.54% 2.36m
Streams: 35.39k
+Resets: 0.05% 18
-Resets: 99.95% 35.37k
Bogus: 0
ZFS Tunable:
metaslab_
zfs_arc_
zfetch_max_streams 8
zfs_nopwrite_
zfetch_
zfs_dbgmsg_enable 0
zfs_dirty_
zfs_arc_
spa_load_
zfs_zevent_cols 80
zfs_dirty_
zfs_sync_
l2arc_write_max 8388608
zfs_vdev_
zfs_vdev_
zvol_prefetch_
metaslab_aliquot 524288
zfs_no_
zfs_arc_
zfetch_block_cap 256
zfs_txg_history 0
zfs_delay_scale 500000
zfs_vdev_
metaslab_
zfs_read_history 0
zvol_max_
zfs_recover 0
l2arc_headroom 2
zfs_deadman_
zfs_scan_idle 50
zfs_free_
zfs_dirty_data_max 1670300876
zfs_vdev_
zfs_mg_
zfs_dedup_prefetch 0
zfs_vdev_
l2arc_write_boost 8388608
zfs_resilver_
zfs_vdev_
zil_slog_limit 1048576
zfs_prefetch_
zfs_resilver_delay 2
metaslab_
zfs_mg_
l2arc_feed_again 1
zfs_zevent_console 0
zfs_immediate_
zfs_dbgmsg_maxsize 4194304
zfs_free_
zfs_deadman_
metaslab_
zfs_arc_
zfs_object_
zfs_metaslab_
zfs_no_scrub_io 0
metaslabs_per_vdev 200
zfs_dbuf_
zfs_vdev_
metaslab_
zvol_inhibit_dev 0
zfs_vdev_
zfs_vdev_
zfs_vdev_
zfs_dirty_
spa_config_path /etc/zfs/
zfs_dirty_
zfs_arc_
zfs_zevent_len_max 128
zfs_scan_
zfs_arc_sys_free 0
zfs_arc_
zfs_vdev_
zfs_arc_
zfs_max_recordsize 1048576
zfs_vdev_
zfs_vdev_
zfs_arc_meta_limit 0
zfs_vdev_
l2arc_norw 0
zfs_arc_meta_prune 10000
metaslab_
l2arc_nocompress 0
zvol_major 230
zfs_vdev_
zfs_flags 0
spa_asize_
zfs_admin_snapshot 0
l2arc_feed_secs 1
zio_taskq_
zfs_sync_
zfs_disable_
zfs_arc_grow_retry 0
zfs_read_
zfs_vdev_
zfs_vdev_
zfs_scrub_delay 4
zfs_delay_
zfs_free_
zfs_vdev_cache_max 16384
zio_delay_max 30000
zfs_top_
spa_slop_shift 5
zfs_vdev_
spa_load_
spa_load_
l2arc_noprefetch 1
zfs_vdev_scheduler noop
zfs_expire_
zfs_sync_
zil_replay_disable 0
zfs_nocacheflush 0
zfs_arc_max 0
zfs_arc_min 0
zfs_read_
zfs_txg_timeout 5
zfs_pd_bytes_max 52428800
l2arc_
zfs_send_
l2arc_feed_min_ms 200
zfs_arc_meta_min 0
zfs_arc_
zfetch_array_rd_sz 1048576
zfs_autoimport
zfs_arc_
zio_requeue_
zfs_vdev_
zfs_mdcomp_disable 0
zfs_arc_
Both tools are well-documented and distributed upstream, see
https:/
https:/
Changed in zfs-linux (Ubuntu): | |
status: | Confirmed → In Progress |
importance: | Undecided → Medium |
assignee: | nobody → Colin Ian King (colin-king) |
Changed in zfs-linux (Ubuntu): | |
assignee: | Colin Ian King (colin-king) → Eric Desrochers (slashd) |
description: | updated |
description: | updated |
Changed in zfs-linux (Ubuntu Xenial): | |
status: | New → In Progress |
importance: | Undecided → Medium |
assignee: | nobody → Eric Desrochers (slashd) |
description: | updated |
description: | updated |
tags: | added: patch sts |
tags: | added: verification-done |
I think it is safe to "assume" python for the purposes of "is this pulling in more code?". It's not safe to "assume" python for the purposes of the actual dependencies in debian/control. That is, python will have to be listed as a dependency, but I don't think adding this dependency to zfsutils-linux is a problem.
A few questions/comments:
1) Does arc_summary.py actually use these binaries, as listed in the comment at the top of the file? I couldn't find any direct calls to them.
# Binaries used are:
#
# dc(1), kldstat(8), sed(1), sysctl(8) & vmstat(8)
2) We need man pages for these commands. (Debian/Ubuntu Policy 12.1). I don't think upstream ships them, so we'll have to write them first.
3) The commands should have the .py extensions dropped, at least when they're installed in the PATH on Ubuntu. (Debian/Ubuntu policy 10.4.)
4) Upstream also has dbufstat. Is that worth shipping too?