MovieTextures: Support for alpha channel RGBA and other color formats

[Kam]

Panda3D version: 1.10.0pre (git 34068dc)
OS: Windows 7 64bit

According to https://github.com/panda3d/panda3d/blob/master/panda/src/ffmpeg/ffmpegVideoCursor.cxx#L518 , Panda3D supports only RGB format decoded on the CPU by FFmpeg for MovieTextures. Movies with an alpha channel (e. g. RGBA format) are not supported yet. A workaround to this issue could be to use a second movie that is greyscale and combine it via TextureStage.setMode, however, this would mean that 2x RGB data is sent from the CPU to the graphics card.

In order to allow an adequate and straightforward way to support videos with alpha channel, RGBA needs to be implemented. Additionally, to save further bandwidth on the PCI express bus and allow higher resoulution video streams and/or more simultanious playing MovieTextures, other formats such as YUV420 and YUVA420 could be implemented to be sent out to the graphics card. Implementing retaining the source format and sending that to the graphics card could allow resolving the alpha channel issue together.

Reproduction:
1. Compile Panda3D with a recent FFmpeg build with libvpx (a recent build from https://ffmpeg.zeranoe.com/builds/ works)

2. Download the attached AlphaChannelPNG.zip and convert it to a video file with alpha channel. E. g.:
ffmpeg -framerate 25 -i AlphaChannel%03d.png -c:v libvpx-vp9 -b:v 2000k -pix_fmt yuva420p AC.webm

3. rename AC.webm to AC.mov, so that Panda regocnizes the file being a video.

4. run pview with the video file, e. g.: pview AC.mov

Expected: You'll see a red box floating around in front of the grey background.
Issue: You'll see the red box in front of a black background. The area that is rendered black is transparent in the video file. For comparison you could run pview with one of the source PNG files.

[Kam (kamg-deactivatedaccount)]

This comment adds the converted and renamed AC.mov from step 2 to this report. The file is encoded in VP9 in 'yuva420p' format by FFmpeg.

[rdb (rdb)]

Could you describe your use case for video playing? Is it something you need to have inside the 3D scene or something you are just playing in a region on screen?

I ask because implementation may be significantly easier depending on how the videos are going to be presented.

[Kam (kamg-deactivatedaccount)]

I need it inside the 3D scene.

My use case: I'm creating a 360°-video-like player, cubic using six faces, similar to a skybox. Each face plays a high resolution video. Around the cube can be another cube, again with video-textures on its faces. To be able to see through the next cube, the foreground videos will have (semi-)transparent areas via their alpha channel.

[rdb (rdb)]

Do you need advanced state settings on the player, aside from transparency? Like multitexturing, lighting, fog, tex offset, tinting, etc? I ask because any shader we make to render these videos would have to implement all of the advanced state that the object will have.

Do you need alpha testing - ie. does the video have large areas with zero opacity, or will most of the video be opaque and semitransparent?

[Kam (kamg-deactivatedaccount)]

Apart from transparency I do not need any advanced features at the moment.

Regarding alpha-testing, I'll try to crop the videos to reduce the alpha surface as much as possible, but it will depend on the asset.

[rdb (rdb)]

I'm going to close this as fixed in 1.10 since 1.10 supports RGBA videos. If you still need the YUV420 support, please file a feature request on GitHub for that.