Comment 4 for bug 1780348

Jay Vosburgh (jvosburgh) wrote :

James,

First, yes, IPv6 has separate neighbour table parameters:

net.ipv4.neigh.default.gc_thresh1 = 128
net.ipv4.neigh.default.gc_thresh2 = 512
net.ipv4.neigh.default.gc_thresh3 = 1024

net.ipv6.neigh.default.gc_thresh1 = 128
net.ipv6.neigh.default.gc_thresh2 = 512
net.ipv6.neigh.default.gc_thresh3 = 1024

Second, give a choice between your two possible sets, I would suggest the second (higher values) set. Explanation to follow:

gc_thresh3 is the absolute limit for the table size, but gc_thresh2 is also important, as when the table size exceeds gc_thresh2 the kernel will aggressively prune entries (deleting entries older than 5 seconds) which could lead to large volumes of ARP or ndisc traffic in pathological situations.

The correct setting for these is ultimately workload dependent, and changing the baked-in kernel default to very large values makes little sense for the typical desktop install, for example. As described in a prior comment, the values may be tuned via cloud-init, so there seems to be no reason to modify the kernel defaults.

I would recommend setting gc_thresh2 to a value equal to the maximum expected number of neighbour entries for a system, plus some head room. The head room can be quite generous (25-50%), as there is no cost to unused capacity in the table. A single neighbour entry is roughly 0.5 KB, so there is minimal risk of excessive memory consumption if the table limit is too high (i.e., 10,000 bogus or stale entries is 5 MB of memory).

In this case, the gc_thresh3 should be set to a value comfortably above the gc_thresh2 value, e.g., another 25%-50% above gc_thresh2, thinking of it as surge capacity.

If your choice is to bake in either the 1024/2048/4096 or 16384/28672/32768 values, I would probably go with the higher values, with the following caveat, below.

Separately, it may also not be desirable to raise the gc_thresh1 to large values; the function of this setting is that if the table contains fewer than gc_thresh1 entries, the kernel will never remove (time out) stale entries. In a "home network" type of environment, this is a reasonable behavior, but in a cloud environment, if IP -> MAC address mappings change, a high gc_thresh1 may lead to hiccups in IP reachability. On the other hand, if the mappings will never, ever change, a high gc_thresh1 may reduce spurious ARP traffic.