Broken support for MPEG-TS: HDV and AVCHD (.mts/.m2ts) files

[Geekkit]

MPEG transport stream (MPEG-TS) is used for HDV and AVCHD camcorders and as such, Linux users need a way to easily play, edit and see thumbnails of these videos. Video editing on Linux can't get very far when the de facto standard for high-definition video isn't well-supported, especially as more and more cameras are HD out of the box, and the prices for such camcorders are falling.

Most of the problem seems to be from the MPEG-TS demuxer in GStreamer [mpegtsdemux] which needs to be fixed or rewritten to support random access better, among other things.

[Geekkit (arron-ferguson-deactivatedaccount)]

I suppose really this is more of a feature request ...

Hi Def video formats (i.e., MTS) are quickly becoming the de facto standard for video camcorders and as such, Ubuntu users need a way to play, edit and see thumbnails of videos created. Currently a user must perform the following:

1) Download FFMPEG current at: http://ffmpeg.mplayerhq.hu/ffmpeg-checkout-snapshot.tar.bz2
2) Then compile/configure/install by running ./configure, make, make install
3) Extract the audio from the m2ts file to output.wav using mplayer:
    mplayer -vc null -vo null -ao pcm:fast:waveheader:file=output.wav xxx.m2ts
4) Extract the video from the m2ts file to output.avi using ffmpeg
    ffmpeg -i xxx.m2ts -vcodec mpeg2video -sameq -s 1920×1080 -r 23.976 -an -f avi -copyts -benchmark output.avi
5) Create a final .avi file by merging the audio and video using mencoder
    mencoder output.avi -o final.avi -ovc copy -oac copy -audiofile output.wav

This is a lot of extra work to simply view files from your new camcorder. Support therefore is required of:

- MTS container format
- H.264 CODEC (including PAFF interlacing, current FFMPEG in Ubuntu gives this error when tying to do the above without downloading the current FFMPEG: "[h264 @ 0xb7e8fa68]PAFF interlacing is not implemented" )
- 1920 X 1080 resolution

The support for this would be required to offer Ubuntu users the following features:
- See thumbnails of their videos in Nautilus (just like they do with other formats such as AVI, MPEG, MOV, OGV, etc.)
- Play/view/hear MTS files in Totem (both video and audio)

This would require incorporating bleeding edge FFMPEG into the distro. It would be the next great step towards multimedia ubiquity within Ubuntu (the next being a NLE app that works ... but that's another issue altogether ;)

[Pedro Villavicencio (pedro)]

that should be incorporated into gstreamer that's not a totem task, reassigning.

[Geekkit (arron-ferguson-deactivatedaccount)]

Pedro Villavicencio,

Thank you kindly reassigning this to the appropriate task, much appreciated.

[Reinhard Tartler (siretart)]

even the ffmpeg version shipped in jaunty does support some variants of AVCHD, AFAIU the upstream bugtracker. Possibly for some specific camcorders some bugfixes need to be backported. Please file propoer bugreports with samples, so that we can test and reproduce the problem. Then a patch against the ubuntu package can be created.

See http://ffmpeg.org/bugreports.html for hints how to create propoer samples and bugreports.

[Jonathan Harker (jonathanharker)]

+1 for VC-1 support, as this will cover Bluray content as well.

[Troy James Sobotka (troy-sobotka)]

I'd add the fact that certain AVCHD streams in MTS wrappers play fine.

What does it take to add the .MTS to totem-video-thumbnailer?

As it is now, as long as it isn't a 24p encoded into a 60i stream, most of the MTS seem to be handled properly by Totem.

[Reinhard Tartler (siretart)]

my understanding of this issue is that ffmpeg in general does support AVCHD files, but there might very well be some producers that produce files that ffmpeg does not understand. If you find such a file, I'd suggest to follow the bug submission guidelines at http://ffmpeg.org/bugreports.html

[Geekkit (arron-ferguson-deactivatedaccount)]

I'm confused as to the status of this bug report. Here are the steps AFAIU them:

1) When I filed this bug with details of what the problem was and how to circumvent it, it was originally filed against Nautilus.

2) Then the bug got reassigned to gstreamer where its state went from "new" to "confirmed"

3) Then (still in gstreamer) "new" to "unknown"

4) Then (still in gstreamer) "unknown" to "new"

5) Suggestion of FFMPEG supporting these files

6) ffmpeg changed from "new" to "fix released"

Does this mean Ubuntu developers are now considering this bug now fixed?

[Reinhard Tartler (siretart)]

Geekkit <email address hidden> writes:

> I'm confused as to the status of this bug report. Here are the steps
> AFAIU them:
>
> 1) When I filed this bug with details of what the problem was and how to
> circumvent it, it was originally filed against Nautilus.

Which is clearly wrong, as nautilus does not decode any media files.

> 2) Then the bug got reassigned to gstreamer where its state went from
> "new" to "confirmed"

which is more correct, because nautilus is using the gstreamer framework
for media files.

> 3) Then (still in gstreamer) "new" to "unknown"
>
> 4) Then (still in gstreamer) "unknown" to "new"

upstream tasks are generally tracked upstream.

> 5) Suggestion of FFMPEG supporting these files

gstreamer does TTBOMK have no AVCHD decoder, but uses ffmpeg for this.

> 6) ffmpeg changed from "new" to "fix released"
>
> Does this mean Ubuntu developers are now considering this bug now fixed?

I with my ffmpeg package maintainer in debian&ubuntu hat on, changed the
ffmpeg upstream bug-task's status, with the rationale given in comment
#6. As the ffmpeg developers do not follow nor even notice bugs
launchpad, and additionally have clear instructions for reporting such
bugs (which are clearly not followed here), I don't see what we as
ubuntu developers can do about this bug other than asking the submitters
to follow the ffmpeg upstream's bug submission guidelines
(i.e. submitting a proper example file).

--
Gruesse/greetings,
Reinhard Tartler, KeyID 945348A4

[Geekkit (arron-ferguson-deactivatedaccount)]

> As the ffmpeg developers do not follow nor even notice bugs
> launchpad, and additionally have clear instructions for reporting such
> bugs (which are clearly not followed here), I don't see what we as
> ubuntu developers can do about this bug other than asking the submitters
> to follow the ffmpeg upstream's bug submission guidelines
> (i.e. submitting a proper example file).

Pretend you're me for a minute (Joe User) and you think you might have a high level understanding of the process (but you most likely do not). You take the time to file a bug because you see this as a way to contribute to the Ubuntu project.

Then you're told that you did it wrong, your bug is closed and no feedback given as to how to "do it right". Imagine how interested you'll be the next time you see an opportunity to "help make Ubuntu better".

Seriously, how am I suppose to know that ffmpeg developers don't come here? How am I suppose to know _where_ to visit the ffmpeg upstream bug submission site? How am I suppose to know what the ffmpeg upstream bug submission guidelines are?

Anyways, cheers. I gave up on this whole bug a few months ago, stopped donating money to Canonical, got smart and realized that if I want to have proper media support, that I'd need to purchase a Mac.

[Vincent Tsugranes (vtsugranes)]

I'm another user experiencing the same problem. Couldn't there be some process where the bug is forwarded from launchpad to upstream ffmpeg? How am I even supposed to know that ffmpeg is the source of the bug? I open up nautilus, double-click on a video, totem opens and doesn't play. Geekit did as much as should be expected of any user, and was failed by this process breakdown.

[Jae Stutzman (jaebird)]

As a dev/user I happen to agree with the sentiment of the last two authors. Sure Nautilus doesn't have anything to do with this, but that is what the user sees. I remember reading something somewhere that Ubuntu was going to work better with upstream to help solve this very problem.

1. A "user" files a bug he sees and does not necessarily know what package is responsible for said bug.
2. Ubuntu devs (or other midnight hackers) figure out what is "really" wrong and internally assign it to the correct package.
3. Somehow the bugs are transfered upstream with links back to launchpad bug.
4. Upstream fixes (eventually in there own time since we don't own or control them)
5. Launchpad bug is marked fixed with the correct upstream version.
6. Users and Devs rejoice!

Otherwise users are going to be intimidated (which goes against the code-of-conduct IIRC) and leave the project.

...or did i mis-read something somewhere?

Cheers.

[Sebastien Bacher (seb128)]

You didn't read something but work is done mostly by contributors and the number of users requests if far higher that what the ubuntu team can look at in a reasonable timeframe which means sometime users can made a difference and give an hand to send issues at the right place if they want to for example

[Neil Wilson (neil-aldur)]

This bug probably covers the lack of support for AVCHD HD video file extensions - as generated by many HD camcorders.

.MTS and .M2TS file extensions should resolve to mime type video/MP2T

[Daniel Leidert (dleidert-deactivatedaccount)]

*** Bug 24690 has been marked as a duplicate of this bug. ***

[Daniel Leidert (dleidert-deactivatedaccount)]

Patch for video/mp2t attached to Bug 24690.

[Oliver-joos-freenet (oliver-joos-freenet)]

I just tested the new patch attached to http://bugs.freedesktop.org/show_bug.cgi?id=24690#c1

My .ts-Files indeed all start with 0x47. But after the patch is applied, all files that start with 0x47 are "video/mp2t", even if their filenames do not end with ".ts". I have here .ts-files that come together with meta files with same name but different suffix. Some of these meta files incidentally start with 0x47, and hence are treated as videos too.

I'd prefer that files are "video/mp2t" only if they start with 0x47 *AND* their filename ends with '.ts' (or '.mp2t' or '.mpts' or '.mpg' or other suffix that might be common for mpeg transport stream files).

[Daniel Leidert (dleidert-deactivatedaccount)]

Can you test with this pattern please?

<match type="big32" mask="0xff5fff1f" value="0x47400010" offset="0"/>

Regards, Daniel

[Oliver-joos-freenet (oliver-joos-freenet)]

This does not match my .ts. They were recorded on a digital cable receiver and start like this:

00000000 47 40 1f 10 00 7f 80 24 00 00 01 00 00 00 00 27 |G@.....$.......'|
...

If I remove some more bits from your mask then it matches my .ts too:
(I did this without knowledge about mpeg stream standards!)

  <match type="big32" mask="0xff5fe01f" value="0x47400010" offset="0"/>

[Kẏra (thekyriarchy)]

Apparently the problem is with the MPEG-TS demuxer [mpegtsdemux] which was never designed for random access so it needs to be fixed or rewritten.

[Omer Akram (om26er)]

Thank you for bringing this bug to our attention. However, a paper cut should be a small usability issue , in the default Ubuntu 9.10 install , that affects many people and is quick and easy to fix. So this bug can't be addressed as part of the project.

For further info about papercuts criteria , pls read > https://wiki.ubuntu.com/PaperCut

Don't worry though, This bug has been marked as "invalid" ONLY in the papercuts project.

[Jeff Fortin Tam (kiddo)]

The title and description of this bug is, from what I can tell, wrong. AVCHD is *not* MPEG-2 TS, and certainly not HDV. Take a look at http://en.wikipedia.org/wiki/AVCHD ; it's all H.264 AVC there.

Thus the current bug report should be about "MPEG-2 Transport Stream" camcorders, because actual *AVCHD* camcorders (at least mine, which does H.264 in a .mov container) do not cause any problems and work fine with PiTiVi.

[Mikko Rantalainen (mira)]

Jean-François Fortin Tam: AVCHD is about H.264 codec but the codec output is stored inside "an MPEG transport stream and stored on media as binary files" (as explained on the wikipedia page you linked to). See http://en.wikipedia.org/wiki/AVCHD#Specifications for details, notice especially the part "Stream type: MPEG transport stream".

A camcorder that stores the H.264 codec output in a .mov container is not AVCHD conforming, if I've understood correctly.

H.264 codec works already fine with gstreamer, the problem is M2TS (or .MTS in 8.3 file system) container format used to store the H.264 codec output. VLC is already able to play H.264 stored inside M2TS container modulo some bugs (such as not being able to correctly skip in the middle of the file).

[Mikko Rantalainen (mira)]

Jean-François Fortin Tam: You're right that HDV is a different format from AVCHD. On the other hand, the letters "HDV" in the summary seems to be short for "high definition video" contrary to HDV format which officially does not have an explanation for the letters. I agree that the use of letters "HDV" in this context is ambiguous and the summary could do better without it.

[Jeff Fortin Tam (kiddo)]

Indeed, my bad for misreading that wikipedia page :)

[Kẏra (thekyriarchy)]

Updated title and description for clarity

[Marcelo Fernandez (fernandezm)]

Well, I have movies recorded with a Panasonic Camera (.MTS extension) and I have some issues (Karmic x86 and x86_64, two very different machines):

- Nautilus doesn't generate a preview thumbnail. I did some mime-type hack (I can't remember what exactly) and now Nautilus does thumbnails. I don't know if this is fixed in Lucid.
- Totem (Gstreamer) plays MTS files very good, but with annoying A/V sync problems (the video has 1 sec. or some of delay). I haven't found a bug of this (should I create one?).
- VLC plays MTS fine too, synchronization is ok, but I see some artifacts, which seems are fixed in trunk [1].

[1] http://trac.videolan.org/vlc/ticket/3277

Hope this helps.

Regards

[Edward Hervey (bilboed) (bilboed-deactivatedaccount)]

To remove (hopefully) some confusion from (amongst other things) the GStreamer side of things.

* HDV/AVCHD in this context means the camera with the 'official' HDV or AVCHD specifications. People linked to those above. Both those formats use mpeg-ts as the container format. If your camera does not record to mpeg-ts... it's not an HDV- or AVCHD- compliant camera.
* The problems aren't that you can't playback the files, but that the support isn't pristine.
* HDV files *playback* fine with latest GStreamer module releases.
* most AVCHD files *playback* fine with latest GStreamer module releases.

The problem here is 'perfect'/'fast' seeking support for those two variants of mpeg-ts. The current gstreamer mpeg-ts demuxer (mpegtsdemux in gst-plugins-bad) was never designed with random-access in mind (it was developed for DVB/streaming support), and that's the root of the issue with those files.

The HDV/AVCHD specifications extend the mpeg-ts usage in very subtle ways making the poor seeking handling in mpegtsdemux even more visible (and in some cases creating A/V sync issues :( ).
Those issues have been known for over a year... but mpegtsdemux is alas tricky code to meddle with and past efforts have only resulted in poor man's seeking capability.

An attempt was started by Zaheer Merali and myself to create a mpeg-ts demuxer from scratch reusing the, much cleaner, mpeg-ts parse codebase which takes into account all those issues and provide a base class for all mpeg-ts variants/use-cases. Problem is... this is low on our respective tasklists :(

P.S. And yes, poor seeking support => reallllly bad support of HDV/AVCHD in PiTiVi and any other gst-based applications who need proper seeking behaviour in mpeg-ts.

[Martin Olsson (mnemo)]

I recently bought a Canon Legria HF200 which I'm using with Ubuntu for family video and some YouTube stuff, it produces AVCHD like most other HD cams which means I get a .mts file with H.264 video codec and on my cam also AC3 audio codec. I've been successfully using this cam to upload youtube videos by first converting the .MTS files to a variant of .mkv using the ffmpeg tool. What I do is this:

ffmpeg -deinterlace -i file_from_cam.mts -sameq -f matroska -vcodec libx264 -acodec vorbis file_that_I_upload_to_youtube.mkv

Of course, it would be so much more user friendly and efficient to have support for these .MTS files right in GStreamer so I could load my clips straight into PiTiVi etc.

[Oliver-joos-freenet (oliver-joos-freenet)]

Created an attachment (id=33748)
Adds support for video/mp2t (MPEG-2 transport streams)

I have here the patch to recognize video/mp2t (MPEG-2 transport streams). It's an enhanced version of https://bugs.freedesktop.org/show_bug.cgi%3Fid%3D24690%23c1. Naming comes from http://www.rfc-editor.org/rfc/rfc3555.txt (page 38). The glob patterns come from http://en.wikipedia.org/wiki/MPEG_transport_stream and are all not yet mentioned in the mime database, except *.ts and *.m2t.

To control matching order of magic patterns one may use "magic priority=", but glob patterns match in order of their appearance in the database file, so:

*.ts is currently recognized as application/x-linguist but without any magic pattern. If we position video/mp2t AFTER it, we can control with the magic pattern if a *.ts becomes video/mp2t or not. I carefully chose the magic pattern to be exactly as strict as necessary to match all valid video/mp2t.

*.m2t is currently recognized as video/mpeg which is wrong according to Wikipedia and e.g. https://bugs.launchpad.net/bugs/89543. So I chose to position video/mp2t BEFORE video/mpeg.

[Stanislav Brabec (sbrabec-suse)]

Maybe this is be better:

+ <_comment>MPEG-2 Transport Stream</_comment>
+ <acronym>MPEG-2 TS</acronym>
+ <expanded-acronym>Moving Picture Experts Group 2 Transport Stream</expanded-acronym>

It is not wise to use the same acronym for two different MIME types.

[Oliver-joos-freenet (oliver-joos-freenet)]

Created an attachment (id=33767)
Adds support for video/mp2t (MPEG-2 transport streams) + unique acronym

@Stanislav: Thanks! I don't know where these acronyms are used, but it is definitely better to keep them unique. So I replaced <acronym> and <expanded-acronym>.

The lower-case letters in <_comment> were chosen intentionally, because other mime types are like this. Probably it does not matter - this attribute is later mapped to a translated string anyway.

[Oliver-joos-freenet (oliver-joos-freenet)]

Please verify patch.

[Oliver-joos-freenet (oliver-joos-freenet)]

Created an attachment (id=34447)
Patch to add support for video/mp2t (MPEG-2 transport streams) + unique acronym + unscrambled only

Patch has been improved. I added another 2 bits to the magic mask to only match if a transport stream is not scrambled. This is ok because scrambled streams could not be handled by common desktop apps anyway. With this mask GIMP resources don't match anymore. (They start with the 4 bytes "GIMP" which is a valid MP2T header)

If you own MP2T files (from a camcorder or a settop box), please test if they match and leave a comment! (Feel free to ask me how to apply the patch to your system)

For details about successful tests see comment 7+8 on https://bugs.launchpad.net/ubuntu/+source/shared-mime-info/+bug/502642

[Oliver-joos-freenet (oliver-joos-freenet)]

Please add this patch to shared-mime-info. Otherwise other bugs are blocked:
https://bugzilla.gnome.org/show_bug.cgi?id=614422
http://trac.videolan.org/vlc/ticket/3485
http://bugs.xine-project.org/show_bug.cgi?id=341

The patch has been successfully applied for Ubuntu 10.04 (see URL in bug details)

[Ville Skyttä (vskytta)]

Current git now has magic for linguist files, could you check if that causes any needs for changes to this patch?

A (small as sanely possible) test file would be welcome for the test suite, do you happen to have one available?

[Oliver-joos-freenet (oliver-joos-freenet)]

I checked new magic for linguist type in current git and saw no problem. The new magic alone does not prevent video/mp2t files to match linguist type. But it does not interfere with my patch either. Together with my patch video/mp2t still are recognized reliably.

I did not check if true application/x-linguist files are affected by my patch, because I have none. But I am pretty sure that this is not the case, because the magic in my patch is quite restrictive. And to verify this I pulled the source of shared-mime-info (including all its test-files) into an Ubuntu Live-CD and then I did:

  $ sudo find /bin /boot /etc /home /lib /opt /root /sbin /tmp /usr /var \
     -xdev -type f -exec mimetype \{\} \; | grep video/mp2t

This showed no false positives.

For a small video/mp2t test file see https://bugs.launchpad.net/ubuntu/+source/shared-mime-info/+bug/502642/comments/7.

[Ville Skyttä (vskytta)]

There's a linguist test file in git. Adding this to tests/list (at end of the video files section) and the mp2t test file in place makes the test suite pass:

test-mp2t.ts video/mp2t xo

The test file seems to be taken from copyrighted material. It's just a couple of seconds but I suppose it'd be better to have a non-copyrighted one there if someone can produce one e.g. with a camcorder.

[Oliver-joos-freenet (oliver-joos-freenet)]

Created an attachment (id=34779)
non-copyrighted mpeg2 transport stream recorded by a settopbox

Thanks for testing. I agree that only non-copyrighted streams should be used as test files. I recorded a video/mp2t snippet with black picture and silent audio. This attachment should be ok to be included in shared-mime-info.

[Oliver-joos-freenet (oliver-joos-freenet)]

Created an attachment (id=35043)
non-copyrighted mpeg2 transport stream recorded by a settopbox

The previous attachment was sub-optimal. It shows is black image sequence but indeed is a non-decrypted content. Now I found a test channel with a nice test pattern (color stripes without any letters or logos). Therefore I submit a new attachment which is IMHO well-suited for inclusion into the source tree of shared-mime-info.

[Jeff Fortin Tam (kiddo)]

In pitivi this is also known as https://bugzilla.gnome.org/show_bug.cgi?id=609601 but I don't know how to link multiple bug watches on launchpad.

[Ville Skyttä (vskytta)]

commit e88b56dd9836c36124baf69d88ae22be8ef776b1
Author: Oliver Joos <email address hidden>
Date: Fri May 21 20:06:31 2010 +0300

    Add video/mp2t.

    http://bugs.freedesktop.org/show_bug.cgi?id=14276

[Mikko Rantalainen (mira)]

May be related but not a duplicate: bug 502642
https://bugs.launchpad.net/bugs/502642

[tekstr1der (tekstr1der)]

Originally this bug's summary was: Missing support for MTS in Totem and Nautilus (thumbnails).

Is this bug still tracking *.mts file thumbnail support in nautilus or is there a separate bug for that issue? Nautilus 2.30.1 in Lucid is still not generating thumbnails for these files. Also, this bug seems to have morphed into pitivi-related concerns.

[madbiologist (me-again)]

Playback of these files should work with the newly released GStreamer Base Plug-ins 0.10.31. From the release notes:

Features of this release

* typefinding: more reliable mpeg-ts typefinding

Bugs fixed in this release

* 623663 : [typefinding] mpeg-ts file detected as audio/mpeg

That's a GNOME bug number - the corresponding bug is at https://bugzilla.gnome.org/show_bug.cgi?id=623663

This package should soon make it's way into the Ubuntu 11.04 "Natty Narwhal" repositories (you can monitor this at https://launchpad.net/ubuntu/+source/gst-plugins-base0.10 ). When it does, you are encouraged to download the Ubuntu 11.04 "Natty Narwhal" alpha 1 LiveCD, update to gst-plugins-base0.10 0.10.31 and test.

[Jeff Fortin Tam (kiddo)]

I am skeptical about this. "typefinding" makes it sound like what was
fixed was just detecting the stuff, not making playback smooth and fully
seekable.

[madbiologist (me-again)]

You might be right. Seeking often comes later (as was the case with .mkv and ogg skeleton v.4). Still an improvement though.

On the other hand, the last paragraph in the original description in the GNOME bug seems to indicate that it will work well. Also the commit description in comment 8 of the GNOME bug sounds like it allows these files to be played back by the pre-existing decoder which we are already familiar with.

I don't have any of these files so I'm hoping you can test Totem with the new gst-plugins-base0.10 0.10.31 package when it hits the repos.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[fgr (f-gritsch)]

@janitor: since 11.04 the bug is fixed. Video-Player plays .MTS files synchron with the audio line.
works for me since the update to 11.04

[Paul D Smyth (pauldsmyth)]

If this is fixed then why am I seeing this in Ubuntu 12.04 right now?

[Rémi Denis-Courmont (rdenis)]

Fixed in VLC 1.1.

[madbiologist (me-again)]

@ Paul D Smyth - Thanks for your comment. I changed the gstreamer task to Fix Released based on comment #49 in this bug. Can you check that you have all of the following installed:

gstreamer0.10-ffmpeg
gstreamer0.10-plugins-bad
gstreamer0.10-plugins-ugly

If you have all of the above installed and are still getting the bug can you please provide steps to reproduce. If you can also provide a small example video file that would be even better.