syslog shows its clearly a driver issue (not nm). Maybe test latest drivers in jaunty  or intrepid-proposed.

from syslog posted:
Dec  6 00:54:32 brain2 NetworkManager: <debug> [1228542872.118124] periodic_update(): Roamed from BSSID 00:14:D1:3E:9E:F4 (mcajnet2) to (none) ((none)) 
Dec  6 00:54:38 brain2 NetworkManager: <debug> [1228542878.118114] periodic_update(): Roamed from BSSID (none) ((none)) to 00:14:D1:3E:9E:F4 (mcajnet2) 
Dec  6 00:56:32 brain2 NetworkManager: <debug> [1228542992.195025] periodic_update(): Roamed from BSSID 00:14:D1:3E:9E:F4 (mcajnet2) to (none) ((none)) 
Dec  6 00:56:38 brain2 NetworkManager: <debug> [1228542998.202469] periodic_update(): Roamed from BSSID (none) ((none)) to 00:14:D1:3E:9E:F4 (mcajnet2) 
Dec  6 00:58:32 brain2 NetworkManager: <debug> [1228543112.286427] periodic_update(): Roamed from BSSID 00:14:D1:3E:9E:F4 (mcajnet2) to (none) ((none)) 
Dec  6 00:58:38 brain2 NetworkManager: <debug> [1228543118.290030] periodic_update(): Roamed from BSSID (none) ((none)) to 00:14:D1:3E:9E:F4 (mcajnet2) 
Dec  6 00:59:08 brain2 kernel: [ 4041.023245] ------------[ cut here ]------------
Dec  6 00:59:08 brain2 kernel: [ 4041.023260] WARNING: at /build/buildd/linux-backports-modules-2.6.27-2.6.27/debian/build/build-generic/compat-wireless-2.6/net/mac80211/rx.c:2200 __lbm_cw_ieee80211_rx+0x19f/0x600 [lbm_cw_mac80211]()
Dec  6 00:59:08 brain2 kernel: [ 4041.023268] Modules linked in: af_packet i915 drm binfmt_misc rfcomm sco bridge stp bnep l2cap bluetooth ppdev ipv6 acpi_cpufreq cpufreq_powersave cpufreq_userspace cpufreq_stats cpufreq_conservative cpufreq_ondemand freq_table container pci_slot sbs sbshc iptable_filter ip_tables x_tables sbp2 parport_pc lp parport joydev pcmcia snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm uvcvideo thinkpad_acpi snd_seq_dummy compat_ioctl32 ath_pci wlan ath_hal(P) videodev yenta_socket rsrc_nonstatic rfkill snd_seq_oss v4l1_compat arc4 pcmcia_core psmouse nvram evdev ecb serio_raw iTCO_wdt crypto_blkcipher pcspkr snd_seq_midi iTCO_vendor_support snd_rawmidi ath5k lbm_cw_mac80211 snd_seq_midi_event lbm_cw_cfg80211 snd_seq led_class battery ac video intel_agp output snd_timer snd_seq_device wmi button snd agpgart shpchp pci_hotplug soundcore snd_page_alloc ext3 jbd mbcache sr_mod cdrom ata_generic usbhid hid sd_mod crc_t10dif sg ata_piix ahci pata_acpi ohci1394 ieee1394 libata scsi_mod ehci_h
Dec  6 00:59:08 brain2 kernel: d uhci_hcd usbcore e1000e dock thermal processor fan fuse vesafb fbcon tileblit font bitblit softcursor
Dec  6 00:59:08 brain2 kernel: [ 4041.023487] Pid: 0, comm: swapper Tainted: P        W 2.6.27-9-generic #1
Dec  6 00:59:08 brain2 kernel: [ 4041.023494]  [<c037c4b6>] ? printk+0x1d/0x1f
Dec  6 00:59:08 brain2 kernel: [ 4041.023509]  [<c0131de9>] warn_on_slowpath+0x59/0x90
Dec  6 00:59:08 brain2 kernel: [ 4041.023523]  [<c0118e38>] ? read_hpet+0x8/0x20
Dec  6 00:59:08 brain2 kernel: [ 4041.023533]  [<c014e63b>] ? getnstimeofday+0x4b/0x100
Dec  6 00:59:08 brain2 kernel: [ 4041.023544]  [<c0136976>] ? set_normalized_timespec+0x16/0x90
Dec  6 00:59:08 brain2 kernel: [ 4041.023554]  [<c037e05e>] ? account_scheduler_latency+0xe/0x220
Dec  6 00:59:08 brain2 kernel: [ 4041.023565]  [<c0123454>] ? __enqueue_entity+0xd4/0x100
Dec  6 00:59:08 brain2 kernel: [ 4041.023574]  [<c0124aba>] ? enqueue_entity+0xda/0x2f0
Dec  6 00:59:08 brain2 kernel: [ 4041.023585]  [<f8cd59af>] __lbm_cw_ieee80211_rx+0x19f/0x600 [lbm_cw_mac80211]
Dec  6 00:59:08 brain2 kernel: [ 4041.023614]  [<c012a708>] ? enqueue_task_fair+0x48/0x50
Dec  6 00:59:08 brain2 kernel: [ 4041.023624]  [<c02eae1c>] ? skb_put+0xc/0xa0
Dec  6 00:59:08 brain2 kernel: [ 4041.023635]  [<c0120f97>] ? enqueue_task+0x57/0x70
Dec  6 00:59:08 brain2 kernel: [ 4041.023647]  [<f8c8ac83>] ath5k_tasklet_rx+0x2e3/0x550 [ath5k]
Dec  6 00:59:08 brain2 kernel: [ 4041.023664]  [<c01136c0>] ? lapic_next_event+0x20/0x30
Dec  6 00:59:08 brain2 kernel: [ 4041.023674]  [<c014e63b>] ? getnstimeofday+0x4b/0x100
Dec  6 00:59:08 brain2 kernel: [ 4041.023686]  [<c0137258>] tasklet_action+0x78/0x100
Dec  6 00:59:08 brain2 kernel: [ 4041.023694]  [<c0137682>] __do_softirq+0x92/0x120
Dec  6 00:59:08 brain2 kernel: [ 4041.023702]  [<c013776d>] do_softirq+0x5d/0x60
Dec  6 00:59:08 brain2 kernel: [ 4041.023710]  [<c01378e5>] irq_exit+0x55/0x90
Dec  6 00:59:08 brain2 kernel: [ 4041.023717]  [<c0106c1a>] do_IRQ+0x4a/0x80
Dec  6 00:59:08 brain2 kernel: [ 4041.023726]  [<c0105003>] common_interrupt+0x23/0x30
Dec  6 00:59:08 brain2 kernel: [ 4041.023735]  [<c01700d8>] ? __audit_mq_getsetattr+0x68/0xb0
Dec  6 00:59:08 brain2 kernel: [ 4041.023747]  [<f899d800>] ? acpi_idle_enter_bm+0x268/0x2b7 [processor]
Dec  6 00:59:08 brain2 kernel: [ 4041.023766]  [<c02dbf7b>] cpuidle_idle_call+0x7b/0xd0
Dec  6 00:59:08 brain2 kernel: [ 4041.023775]  [<c010288d>] cpu_idle+0x7d/0x140
Dec  6 00:59:08 brain2 kernel: [ 4041.023782]  [<c037a711>] start_secondary+0x9d/0xcc
Dec  6 00:59:08 brain2 kernel: [ 4041.023793]  =======================
Dec  6 00:59:08 brain2 kernel: [ 4041.023798] ---[ end trace 6d64d49b0d28393f ]---