I'm unable to play "Transformers 3 - Dark of the Moon" with what looks like a similar failure, whilst Movie Player is able to play the disk (Totem 3.0.1 with 0.10.35 gstreamer on ubuntu).  Version info for vlc and libdvdnav and log below.

cefn@cefn-dell-oneiric:~$ vlc --version
VLC media player 2.0.0 Twoflower (revision 2.0.0-0-g421a4fc)
VLC version 2.0.0 Twoflower (2.0.0-0-g421a4fc)
Compiled by buildd on nannyberry.ppa (Mar  3 2012 01:06:44)
Compiler: gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) 
This program comes with NO WARRANTY, to the extent permitted by law.
You may redistribute it under the terms of the GNU General Public Licence;
see the file named COPYING for details.
Written by the VideoLAN team; see the AUTHORS file.

Here's a console log from the GUI when I try to open the DVD direct (similar experience with an iso image). I upgraded to a newer VLC version using ppa:bdrung/backports. That gives me libdvdnav 4.2.0-1~oneiric1~ppa1 which I've combined with vlc 2.0.0-6~ppa1 . Trying to play the video gives the log below with the last lines, (or very similar lines) seeming to run forever and ever. 

cefn@cefn-dell-oneiric:~$ vlc 2>&1 | head -n 100
[0x9450920] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
libdvdnav: Using dvdnav version 4.2.0
libdvdread: Using libdvdcss version 1.2.11 for DVD access
libdvdnav: DVD Title: TRANSFORMERS_3
libdvdnav: DVD Serial Number: 3F298BD7
libdvdnav: DVD Title (Alternative): TRANSFORMERS_3
libdvdnav: Unable to find map file '/home/cefn/.dvdnav/TRANSFORMERS_3.map'
libdvdnav: DVD disk reports itself with Region mask 0x00ed0000. Regions: 2 5

libdvdread: Attempting to retrieve all CSS keys
libdvdread: This can take a _long_ time, please be patient

libdvdread: Get key for /VIDEO_TS/VIDEO_TS.VOB at 0x0000020b
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_01_1.VOB at 0x000006f4
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_02_1.VOB at 0x00000712
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_03_1.VOB at 0x00000761
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_04_1.VOB at 0x0000077f
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_05_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_05_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_06_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_06_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_07_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_07_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_08_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_08_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_09_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_09_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_10_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_10_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_11_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_11_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_12_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_12_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_13_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_13_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_14_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_14_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_15_0.VOB at 0x00000ac8
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_15_1.VOB at 0x000067bf
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_16_0.VOB at 0x00384a17
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_16_1.VOB at 0x00387171
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_17_0.VOB at 0x0038718e
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_17_1.VOB at 0x003871fe
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_18_1.VOB at 0x00387de4
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_19_1.VOB at 0x00387e00
libdvdread: Elapsed time 0
libdvdread: Found 19 VTS's
libdvdread: Elapsed time 0

*** libdvdread: CHECK_VALUE failed in /build/buildd/libdvdread-4.1.3/src/ifo_read.c:1146 ***
*** for data[i] + sizeof(ptt_info_t) <= vts_ptt_srpt->last_byte + 1 + 4 ***


*** libdvdread: CHECK_VALUE failed in /build/buildd/libdvdread-4.1.3/src/ifo_read.c:1146 ***
*** for data[i] + sizeof(ptt_info_t) <= vts_ptt_srpt->last_byte + 1 + 4 ***


*** libdvdread: CHECK_VALUE failed in /build/buildd/libdvdread-4.1.3/src/ifo_read.c:1146 ***
*** for data[i] + sizeof(ptt_info_t) <= vts_ptt_srpt->last_byte + 1 + 4 ***


*** libdvdread: CHECK_VALUE failed in /build/buildd/libdvdread-4.1.3/src/ifo_read.c:1146 ***
*** for data[i] + sizeof(ptt_info_t) <= vts_ptt_srpt->last_byte + 1 + 4 ***


*** libdvdread: CHECK_VALUE failed in /build/buildd/libdvdread-4.1.3/src/ifo_read.c:1146 ***
*** for data[i] + sizeof(ptt_info_t) <= vts_ptt_srpt->last_byte + 1 + 4 ***