Didn't take long:

[ 2111.864905] WARNING: at /home/indan/src/linux-2.6/drivers/char/agp/intel-gtt.c:1007 intel_i830_chipset_flush+0x2e3/0x32d()
[ 2111.864912] Hardware name: 2371GHG
[ 2111.864917] i8xx chipset flush failed, expected: 118451, cpu_read: 117939
[ 2111.864922] Modules linked in: pl2303 usbserial usb_storage uhci_hcd ehci_hcd usbcore
[ 2111.864940] Pid: 788, comm: X Not tainted 2.6.34-rc6-v9 #52
[ 2111.864945] Call Trace:
[ 2111.864956]  [<c101ea80>] ? warn_slowpath_common+0x5d/0x70
[ 2111.864964]  [<c101eac6>] ? warn_slowpath_fmt+0x26/0x2a
[ 2111.864973]  [<c1141b1b>] ? intel_i830_chipset_flush+0x2e3/0x32d
[ 2111.864984]  [<c113da68>] ? agp_flush_chipset+0xc/0xd
[ 2111.864994]  [<c115bdae>] ? i915_gem_flush+0x1a/0xbb
[ 2111.865003]  [<c115fae5>] ? i915_gem_do_execbuffer+0x9bb/0xe3f
[ 2111.865023]  [<c115d187>] ? i915_gem_object_set_to_gtt_domain+0x33/0x5c
[ 2111.865032]  [<c116004d>] ? i915_gem_execbuffer2+0xe4/0x164
[ 2111.865041]  [<c114826f>] ? drm_ioctl+0x1cf/0x27a
[ 2111.865049]  [<c115ff69>] ? i915_gem_execbuffer2+0x0/0x164
[ 2111.865060]  [<c10695d9>] ? do_sync_read+0x9d/0xd2
[ 2111.865069]  [<c11480a0>] ? drm_ioctl+0x0/0x27a
[ 2111.865078]  [<c1073701>] ? vfs_ioctl+0x1c/0x7d
[ 2111.865086]  [<c1073c97>] ? do_vfs_ioctl+0x478/0x4bc
[ 2111.865096]  [<c1030e62>] ? hrtimer_try_to_cancel+0x43/0x60
[ 2111.865105]  [<c1021c12>] ? do_setitimer+0xa4/0x17f
[ 2111.865113]  [<c1021d35>] ? sys_setitimer+0x48/0x73
[ 2111.865121]  [<c1034f41>] ? ktime_get_ts+0xb3/0xbb
[ 2111.865129]  [<c1073d08>] ? sys_ioctl+0x2d/0x44
[ 2111.865138]  [<c10025d0>] ? sysenter_do_call+0x12/0x26
[ 2111.865144] ---[ end trace d90ca0d623dcc2a3 ]---
[ 2934.051532] ------------[ cut here ]------------
[ 2934.051547] WARNING: at /home/indan/src/linux-2.6/drivers/char/agp/intel-gtt.c:1007 intel_i830_chipset_flush+0x2e3/0x32d()
[ 2934.051551] Hardware name: 2371GHG
[ 2934.051554] i8xx chipset flush failed, expected: 156295, cpu_read: 155783
[ 2934.051557] Modules linked in: pl2303 usbserial usb_storage uhci_hcd ehci_hcd usbcore
[ 2934.051569] Pid: 788, comm: X Tainted: G        W  2.6.34-rc6-v9 #52
[ 2934.051572] Call Trace:
[ 2934.051580]  [<c101ea80>] ? warn_slowpath_common+0x5d/0x70
[ 2934.051584]  [<c101eac6>] ? warn_slowpath_fmt+0x26/0x2a
[ 2934.051589]  [<c1141b1b>] ? intel_i830_chipset_flush+0x2e3/0x32d
[ 2934.051596]  [<c113da68>] ? agp_flush_chipset+0xc/0xd
[ 2934.051602]  [<c115bdae>] ? i915_gem_flush+0x1a/0xbb
[ 2934.051607]  [<c115fae5>] ? i915_gem_do_execbuffer+0x9bb/0xe3f
[ 2934.051614]  [<c116620f>] ? intel_mark_busy+0x9b/0x177
[ 2934.051619]  [<c115d187>] ? i915_gem_object_set_to_gtt_domain+0x33/0x5c
[ 2934.051624]  [<c116004d>] ? i915_gem_execbuffer2+0xe4/0x164
[ 2934.051629]  [<c114826f>] ? drm_ioctl+0x1cf/0x27a
[ 2934.051634]  [<c115ff69>] ? i915_gem_execbuffer2+0x0/0x164
[ 2934.051641]  [<c1007a62>] ? restore_i387_fxsave+0x4c/0x5c
[ 2934.051647]  [<c1034fa4>] ? ktime_get+0x5b/0xcf
[ 2934.051652]  [<c11480a0>] ? drm_ioctl+0x0/0x27a
[ 2934.051658]  [<c1073701>] ? vfs_ioctl+0x1c/0x7d
[ 2934.051662]  [<c1073c97>] ? do_vfs_ioctl+0x478/0x4bc
[ 2934.051669]  [<c1031655>] ? hrtimer_start+0xd/0x11
[ 2934.051674]  [<c1021c91>] ? do_setitimer+0x123/0x17f
[ 2934.051678]  [<c1034f41>] ? ktime_get_ts+0xb3/0xbb
[ 2934.051683]  [<c1073d08>] ? sys_ioctl+0x2d/0x44
[ 2934.051687]  [<c10025d0>] ? sysenter_do_call+0x12/0x26
[ 2934.051691] ---[ end trace d90ca0d623dcc2a4 ]---

There's also a small copy&paste bug in your patch:

	for (i = 0; i < I830_CC_CANARY_FLOCK_PAGES; i++) {
		intel_private.i8xx_cpu_canary_pages[i]
			= kmap(intel_private.i8xx_pages[i+2]);
		if (!intel_private.i8xx_cpu_flush_page) {
			WARN_ON(1);
			intel_i830_fini_flush();
			return;
		}
	}
That should be if (!intel_private.i8xx_cpu_canary_pages[i]).

I don't understand this bit:

	/* Don't map the first page, we only write via its physical address
	 * into it. */
	for (i = 0; i < I830_CC_DANCE_PAGES; i++) {
		writel(agp_bridge->driver->mask_memory(agp_bridge,
				page_to_phys(intel_private.i8xx_pages[i+1]), 0),
		       intel_private.registers+I810_PTE_BASE+((num_entries+i)*4));
	}

The first page is i8xx_cpu_flush_page, but if it isn't mapped, the gmch doesn't know about it, and intel_flush_mch_write_buffer() has no effect, has it? Or is any write at any address sufficient to fill the write buffer?

We seem to have mysterious behaviour here, all the canary pages ended up coherent, but that one write somehow didn't?!

I guess the gmch has a local cache that hides writes. If you know that cache's design (associativity etc.) then you can probably flush it out by doing a read or write to the right address. The canary stuff seems to work most of the time, so the cache can't be too big.

Or maybe you can flush it out by putting the chip in D1-3 and back to D0 quickly, or something crazy like that.