Comment 9 for bug 644289

I think i found the problem... at least i'm able to get avidemux to work instead of getting these silly assertions!

The problem is in the avidemux source code where they impose arbitrary limits on the number of P-frames that can follow an I-frame ... restrictions that are exceeded by mencoder...

I downloaded source code for avidemux 2.5.5 (Ubuntu 11.10 uses 2.5.4 but i think the problem is the same).
In the directory avidemux_2.5.5/avidemux/ADM_inputs i fixed the following:
ADM_mpegdemuxer/dmx_indexer_internal.h :
changed
 #define MAX_PUSHED 200
to
#define MAX_PUSHED 2000
... and in ADM_mpegdemuxer/dmx_video.cpp
changed
#define MAX_LINE 4096
to
#define MAX_LINE 40960

Of course, there is no guarantee that these sizes are large enough either (although they worked for my case)...
clearly the program algorithm should be rethought to work with arbitrary sizes...
there could be other similar problems in the code as well...

In order to compile beneath ubuntu 11.0 i ALSO had to make the following changes:
in plugins/ADM_videoEncoder/ADM_vidEnc_x264/encoder.cpp
i commented out the line that says:
printf("[x264] i_open_gop = %d\n", x264Param->i_open_gop);
... and in plugins/ADM_videoEncoder/ADM_vidEnc_x264/x264Options.cpp
i modified x264Options::getOpenGopMode(void) to simply return 0 and
modified x264Options::setOpenGopMode(unsigned int openGopMode)
to do nothing (to avoid errors related to undefined field i_open_gop).
... i have no idea if this might cause a problem with someone using x264 stuff...

With these changes i have been able to use my locally compiled avidemux_qt4 under ubuntu 11.10 to edit dvd-compatible mpeg output generated by mencoder...

-Ted Merrill