I found the possible cause of the bug and reported this upstream.
https://bugs.freedesktop.org/show_bug.cgi?id=34710

Quote the bug report on fd.o here so more people can see it.

UDisks failed to perform ForceUnmount when I press the physical eject button on
my CDROM.
So I downloaded udisks source code from git and traced it for a while and then
I found the problem.

Here are the related properties of my device.
--------------------------------------------------------------------------------------------------------------
  device-file:                 /dev/sr0
  removable:                   1
  has media:                   1 (detected at Fri Feb 25 19:16:23 2011)
    detects change:            1
    detection by polling:      1  <---  I have no SATA AN, so polling is
performed.
    detection inhibitable:     1
    detection inhibited:       0  <-- polling of the device is not inhibited.
  is read only:                0
  is mounted:                  1
  drive:
    vendor:                    HL-DT-ST
    model:                     HL-DT-ST DVDRAM GMA-4082N
    revision:                  CX08
    serial:                    M0573JA3106
    detachable:                0
    ejectable:                 1
    media:                     optical_cd
    interface:                 scsi
-------------------------------------------------------------------------------------------------------------
I tested udisks on a clean system in a clean xsession + terminal emulator so
there shouldn't be other programs interfering with udisks.

In src/poller.c, poller_poll_device() :

      fd = open (device_file, O_RDONLY | O_NONBLOCK | O_EXCL);
      if (fd != -1)
          close (fd);

When I inserted a CD, this call to open() succeeded and returned a valid fd. I
enabled POLL_SHOW_DEBUG, and according to the debug message, the device was
polled every 2 seconds as expected.

However, after the device is mounted, either by udisks --mount or by calling
sudo mount myself, open(device_file) always returns -1 here.
Then, I press the physical eject button on my CDROM to get its tray opened. The
polling code should let udisks detect the media removal and then invokes
ForceUnmount, but this is not the case. The call to open() kept returning -1
every 2 seconds and udisks didn't detect the media change at all. Checking
errno, "Device is busy" is reported.

Then I remove the O_EXCL flag and perform the whole test again. This time,
after the CD is removed by pressing physical eject button, the open() call
succeeded and returned a valid fd. Then, udisks detects the media change and
gracefully perform ForceUnmount for it. Everything works fine as expected.

In addition, when "Device is busy", I tried fuser and lsof, but none of them
demonstrates other process using the device. However, if I unmount the device
either by calling sudo umount or udisks --unmount, this error is gone and
open() can return a valid fd. Then everything works.

Removing O_EXCL from this open() call seems to fix the issue, but this of
course is not the correct way to fix it.

Another interesting thing I found is, if I call udisks --poll-for-media change
manually after forced removal of CD with physical eject button, then udisks can
correctly detect media change. The polling helper process uses the same source
code as in poller.c, but poller.c doesn't work and I don't know why.

Please, if any udisks developer is reading this, test this with a CDROM without
SATA AN. It's 100% reproducible here. The poller is invoked every 2 seconds
correctly, but the polling does not work. One of my friend have similar
problems that after physical eject udisks didn't detect the change.