I experienced a similar problem with Ubuntu (Desktop) 9.04 and 9.10 (beta) a few times.
The system has 4GB physical memory and 3,9GB swap-space.

1.) I copied some large files (maybe 20GB each) by gigabit network. After some time, the transfer rate went to near zero, hdd-led stays on with a slight flickering. I klicked to open a konsole window but it did not open. I switched to tty2, typed root, hit enter. It showed a blinking cursor but no password promt appeared. I ssh'd from another computer with same result.
After about 30 minutes, system went back normal.

2.) While watching a video with vlc, i copied a large file from one parition to another (same physical disk). After a few minutes, system becomes unresponsive, stuttering sound, cursor sticks. After some minutes, vlc is terminated. Ten minutes later system becomes normal. I was able to see the memory usage with top. 100% used, both pysical and swap. Apart from vlc and the file copy, there were no other activities.

Here is what the logfiles show:
[12725.291450] padevchooser invoked oom-killer: gfp_mask=0x201da, order=0, oomkilladj=0
[12725.291454] padevchooser cpuset=/ mems_allowed=0
[12725.291457] Pid: 3146, comm: padevchooser Tainted: P        W  2.6.31-14-generic #48-Ubuntu
[12725.291459] Call Trace:
[12725.291466]  [<ffffffff810a2798>] ? cpuset_print_task_mems_allowed+0x98/0xa0
[12725.291471]  [<ffffffff810dd7ce>] oom_kill_process+0xce/0x290
[12725.291474]  [<ffffffff810ddd3a>] ? select_bad_process+0xea/0x120
[12725.291476]  [<ffffffff810dddc0>] __out_of_memory+0x50/0xb0
[12725.291479]  [<ffffffff810ddf46>] out_of_memory+0x126/0x1a0
[12725.291483]  [<ffffffff815293a9>] ? _spin_lock+0x9/0x10
[12725.291486]  [<ffffffff810e0808>] __alloc_pages_slowpath+0x498/0x4e0
[12725.291489]  [<ffffffff810e099e>] __alloc_pages_nodemask+0x14e/0x150
[12725.291493]  [<ffffffff8110cea2>] alloc_pages_current+0x82/0xd0
[12725.291496]  [<ffffffff810da92f>] __page_cache_alloc+0x5f/0x70
[12725.291498]  [<ffffffff810da710>] ? sync_page+0x0/0x50
[12725.291502]  [<ffffffff810e4601>] __do_page_cache_readahead+0xc1/0x160
[12725.291505]  [<ffffffff810e46bc>] ra_submit+0x1c/0x20
[12725.291507]  [<ffffffff810da42b>] do_sync_mmap_readahead+0x9b/0xd0
[12725.291510]  [<ffffffff810dc594>] filemap_fault+0x314/0x3c0
[12725.291513]  [<ffffffff810f423f>] __do_fault+0x4f/0x4e0
[12725.291516]  [<ffffffff810f89d7>] handle_mm_fault+0x1a7/0x3c0
[12725.291520]  [<ffffffff81036419>] ? default_spin_lock_flags+0x9/0x10
[12725.291523]  [<ffffffff8152bf4a>] do_page_fault+0x16a/0x370
[12725.291526]  [<ffffffff815298c5>] page_fault+0x25/0x30
[12725.291528] Mem-Info:
[12725.291530] Node 0 DMA per-cpu:
[12725.291532] CPU    0: hi:    0, btch:   1 usd:   0
[12725.291534] CPU    1: hi:    0, btch:   1 usd:   0
[12725.291536] Node 0 DMA32 per-cpu:
[12725.291538] CPU    0: hi:  186, btch:  31 usd: 146
[12725.291540] CPU    1: hi:  186, btch:  31 usd: 175
[12725.291541] Node 0 Normal per-cpu:
[12725.291544] CPU    0: hi:  186, btch:  31 usd:  83
[12725.291546] CPU    1: hi:  186, btch:  31 usd: 160
[12725.291549] Active_anon:765930 active_file:160 inactive_anon:190722
[12725.291550]  inactive_file:281 unevictable:4 dirty:0 writeback:927 unstable:0
[12725.291551]  free:6600 slab:9537 mapped:3337 pagetables:10621 bounce:0
[12725.291553] Node 0 DMA free:15860kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:15320kB pages_scanned:0 all_unreclaimable? yes
[12725.291558] lowmem_reserve[]: 0 3511 4016 4016
[12725.291562] Node 0 DMA32 free:9556kB min:7076kB low:8844kB high:10612kB active_anon:2876196kB inactive_anon:575140kB active_file:60kB inactive_file:76kB unevictable:0kB present:3595360kB pages_scanned:29 all_unreclaimable? no
[12725.291567] lowmem_reserve[]: 0 0 505 505
[12725.291570] Node 0 Normal free:984kB min:1016kB low:1268kB high:1524kB active_anon:187524kB inactive_anon:187748kB active_file:580kB inactive_file:1048kB unevictable:16kB present:517120kB pages_scanned:2496 all_unreclaimable? no
[12725.291575] lowmem_reserve[]: 0 0 0 0
[12725.291579] Node 0 DMA: 1*4kB 4*8kB 5*16kB 2*32kB 3*64kB 1*128kB 2*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15860kB
[12725.291587] Node 0 DMA32: 853*4kB 6*8kB 1*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 1*4096kB = 9556kB
[12725.291596] Node 0 Normal: 24*4kB 15*8kB 6*16kB 1*32kB 2*64kB 2*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 984kB
[12725.291604] 4344 total pagecache pages
[12725.291606] 2327 pages in swap cache
[12725.291608] Swap cache stats: add 1057178, delete 1054851, find 7927/11280
[12725.291609] Free swap  = 0kB
[12725.291611] Total swap = 4096532kB
[12725.302153] 1048576 pages RAM
[12725.302155] 34276 pages reserved
[12725.302156] 5707 pages shared
[12725.302157] 1003137 pages non-shared
[12725.302160] Out of memory: kill process 31894 (vlc) score 2032821 or a child
[12725.302194] Killed process 31894 (vlc)