OK, now it's clear what is happening. The test assumes that the server running on port 12347 is a "second master" but it's not, it's replicating from the first "master".
If no server is running on port 12347, this works fine -- the test starts one as a standalone server. But if it's already running, it breaks because the test starts making changes to the 12347 instance that conflict with replication. We need another sandbox instance to be a real "master" instead of hijacking a slave and treating it like one.