Comment 34 for bug 705562

Sorry for getting back a bit late. I had been playing with xfs getting mounted and unmounted and also writing to it in between. This did not trigger the bug. Neither did removing the xfs module in between.
Not sure this will lead anywhere but as the dmesg seemed to indicate that at least some modules were loaded again (so must have been unloaded), maybe you could create a little script like

while true; do
  date
  lsmod
  sleep 60
done >modules.log

to track what modules are loaded at which point in time.