It's not supposed to hold all the files in memory. It's supposed to generate a map of hashes to file ids. There is a hard limit of 10 Mi hashes. If multiple copies of the file-id were stored in memory, perhaps that could cause this memory exhaustion, but the file-ids should be interned.
It's not supposed to hold all the files in memory. It's supposed to generate a map of hashes to file ids. There is a hard limit of 10 Mi hashes. If multiple copies of the file-id were stored in memory, perhaps that could cause this memory exhaustion, but the file-ids should be interned.