Unable to diagnose where docker filesystem usage is going
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
StarlingX |
Fix Released
|
Medium
|
Jim Gauld |
Bug Description
Brief Description
-----------------
Cannot diagnose from collect logs where docker file-system usage is going.
The existing data in collect has overall filesystem usage, various crictl commands, and docker images commands, but is not detailed enough to see the remnants from an initial install. This prevents simple debugging of lab issues and field issues when we exceed file-system alarm threshold of 80% for /var/lib/docker.
Desire the addition of the following two commands to collect containerizatio
docker system df
du -h --max-depth 1 /var/lib/docker
If an admin manually does docker operations to pull in images or other, it will show in the historical docker filesystem /var/lib/
e.g.,
-------
Fri Jun 3 21:13:09 UTC 2022 : : docker system df
-------
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 1 0 688.5MB 688.5MB (100%)
Containers 0 0 0B 0B
Local Volumes 0 0 0B 0B
Build Cache 0 0 0B 0B
-------
Fri Jun 3 21:13:09 UTC 2022 : : du -h --max-depth 1 /var/lib/docker
-------
0 /var/lib/
0 /var/lib/
719M /var/lib/
2.9M /var/lib/
24K /var/lib/
0 /var/lib/
32K /var/lib/
0 /var/lib/
16K /var/lib/
56K /var/lib/
0 /var/lib/
1.7G /var/lib/
0 /var/lib/
5.0G /var/lib/
3.9M /var/lib/
0 /var/lib/
12K /var/lib/
664K /var/lib/
0 /var/lib/docker/tmp
0 /var/lib/
7.4G /var/lib/docker
Severity
--------
Major: Cannot debug field issues when we have docker related file-system alarms.
Steps to Reproduce
------------------
Gather a 'collect'.
Expected Behavior
------------------
See detailed first-level usage under /var/lib/docker .
Actual Behavior
----------------
State what is the actual behavior
Reproducibility
---------------
100 percent reproducible.
Always see docker remnants due to initial install via docker from install of n3000-opae.
Occasionally admins will install other stuff.
System Configuration
-------
AIO-DX. Applicable to all configs.
Branch/Pull Time/Commit
-------
BUILD_DATE=
Last Pass
---------
No. Day one issue.
Timestamp/Logs
--------------
Collect /var/extra/
docker image ls -a
crictl images
ctr -n k8s.io images list
Collect /var/extra/
df -h -H -T --local -t ext2 -t ext3 -t ext4 -t xfs --total
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/
Test Activity
-------------
Feature Testing, Evaluation.
Changed in starlingx: | |
assignee: | nobody → Jim Gauld (jgauld) |
tags: | added: stx.7.0 stx.tools |
Changed in starlingx: | |
importance: | Undecided → Medium |
Fix proposed to branch: master /review. opendev. org/c/starlingx /utilities/ +/844844
Review: https:/