Ok, my first proposal was delightfully naive 8-(. Always fun to see something that appears to be easy to have a huge pile of hair..
So, I did some more thinking to try to get a more failsafe solution- and it proves to be quite nasty. I have written down the details as far as I see them now at http://wiki.bazaar.canonical.com/fjalvingh. A short rundown of the conclusions is:
* Any filtering rule change MUST go through the bzr UI. The file should NOT be user-editable at all (there is an alternative but it is more ugly)
* Changing rules is allowed for a non-dirty working tree only
* Changing rules may (must) cause the rewrite of the working tree, forcing those files to be proper for the new rules.
* Some changes must fully-change the repo with a commit. It would be good if the user would be warned when this happens because merging stuff from before that point could be almost impossible..
@Marius: thanks for your answer. I see some trouble with maintaining the rules file there and imposing a normal (file based) merge on it. I don't like the idea of my users mucking around in there- there's all kinds of stuff below .bzr and allowing them to edit there (as they will when there is a conflict) can open cans of worms attracting fishy problems.
The versioned properties thing looks more like it, although the merge rules for the rules file would actually be quite funny if you do it right; it should not just give two copies of a big multi-wildcard spec there. I would be surprised if actually merging files would be a good idea there. It might be best just to merge serialized structures using a special UI.