negative effect gets wrong on some transparent elements

[Samuel thibault]

Hello,

The negative effect seems to be getting applied twice on some transparent parts of the rendering, thus canceling the desired effect on these parts, which poses problems to some users with sight impairements.

I can reproduce it this way:

- Use a MATE desktop
- Install compiz (here using version 0.9.13.1+18.04.20180302) with metacity support enabled
- Enable the decorations plugin
- Enable the negative plugin
- In the negative plugin, enable negating decorations
- Activate negation (super-m)

Then windows will have a thick white border around them (I have attached a screenshot). This is actually the shadow decoration which gets inverted twice instead of just once (changing the decoration shadow width does change the thick white border width).

We also have the same kind of issue in the static application switcher, when transparent parts of icons show up white. This can be better seen by requesting to "show icon only" to make the icons bigger.

I'm not sure how this can easily be fixed, AFAICT what is happening is that there the pixels behind the window gets rendered twice: once normally, and a second time for the decoration shadow, and thus the neg plugin's glDrawTexture function gets triggered each time, thus inverting the original pixel twice. I don't know how glDrawTexture could perhaps distinguish between the two calls, to avoid one of the inversions?

[Samuel thibault (samuel-thibault)]

I forgot to mention why I was thinking the effect gets applied twice: I modified the neg shader to just add 0.2 to a color channel, and noticed that the effect would add up.

But now that I have experimented more with the switcher (which is where the bug is really problematic in our usecase), it seems it's not so simple. This is what I tried:

- Add a PAINT_WINDOW_ADDITIONAL_MASK mask flag to be used when there are several rendering with alpha blending on the same area, only one should not use it, for the neg plugin to know which one should apply the neg shader.
- For a start, use it in the switcher's glPaint call for the switcher popup frame, so that it's not inverted, and only the icon gets inverted.

But the transparent parts of the icon do not get inverted by neg actually: the popup frame has a grey background, and thus inversion should give another grey for the inverted transparent parts, but they just become completely white, as if the rendering of the icon was actually getting black pixels instead of the popup frame background grey color.

I then realized that BaseSwitchWindow::paintThumb uses the original application's gWindow to render the icons. I propagated the gWindow of StaticSwitchWindow::glPaint to paintThumb to use the popup frame's gWindow, but the result is the same, so I must be missing something in how rendering the icon gets the pixels behind its transparency.

But when not enabling the neg plugin (i.e. just not adding the neg shader to the pipeline), the popup frame color does show up properly in transparent areas of the icons. So it'd rather be the neg shader which does not properly get the right pixel to be used, i.e. something else to be used than gl_FragColor?

[Samuel thibault (samuel-thibault)]

Or perhaps for some pixels a different encoding for gl_FragColor?

[Samuel thibault (samuel-thibault)]

Here a sort of workaround: making the neg plugin just not invert pixels whose alpha is less than 0.5. That still produces some whitish frame around icons which have a little antialias alpha border for smoothing them, but the result is way better than what I manage to work out otherwise.