Stamp annotations cannot be printed to pdf or "flattened"

Bug #1859632 reported by Denny
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
okular
Unknown
Medium
okular (Ubuntu)
Undecided
Unassigned

Bug Description

Ubuntu 19.10
Okular Installed: 4:19.04.3-0ubuntu1

What I wanted to do:
I need a pdf file including my signature in a non-editable format.I have created a respective custom annotation. As there is no option to "flatten" annotations, I thought printing to pdf would be a suitable workaround. This workarount works for other annotations, but not for stamps (even for non-custom).

steps to reproduce:
Open a pdf file add a stamp annotation, print to pdf (tick print annotations).

Expected result:
A pdf with non-editable ("flattened") stamp annotations.

Actual result:
A pdf without stamp annotations

Revision history for this message
In , Nate-b (nate-b) wrote :

Okular 1.0.3 on Kubuntu 17.04

A few years ago, Okular gained the ability to save annotations into PDF files: https://bugs.kde.org/show_bug.cgi?id=151614

For the most part, it works great! There is one problem: custom/image stamp annotations are not saved in a way that can be printed or that other PDF readers can see. I tested many.

STEPS TO REPRODUCE:
1. Open a PDF file with Okular
2. Define a custom image stamp (Configure Okular > Annotations > Add > Stamp > enter a valid path to an image in the combobox)
3. Stamp your image into the document
4. File > Save As > Save the document somewhere
5. Use Okular to print the document or open it document in a non-Okular PDF viewer

EXPECTED RESULTS:
The image stamp is visible when the document is printed or viewed in a non-Okular PDF viewer

ACTUAL RESULTS:
The image stamp annotation is not visible In any of the non-Okular PDF viewers I had available: Adobe Acrobat, Foxit Reader, SumatraPDF, the PDF readers built into Firefox and Microsoft Edge. Acrobat reader can see and manipulate the *outlines* of the annotations, but the outlines have no images in them.

ADDITIONAL INFORMATION:
This affects a very common use case: wanting to add an image of the user's signature to a PDF document to sign it and send it back to them. When they open the file, the signature will not be visible unless they open it with Okular (unlikely).

Revision history for this message
In , Nate-b (nate-b) wrote :

Is this caused 100% by https://bugs.freedesktop.org/show_bug.cgi?id=23108? Or are there other things that need to be done if/once poppler adds appearance streams for rubber stamps?

Revision history for this message
In , Nate-b (nate-b) wrote :

Albert, will the printing case be resolved with https://bugs.freedesktop.org/show_bug.cgi?id=102513?

Revision history for this message
In , Nate-b (nate-b) wrote :

(in combination with https://phabricator.kde.org/D7688, of course)

Revision history for this message
In , Oliver-sander (oliver-sander) wrote :

No. Your problem really smells like a problem with annotation handling, and printing is just a symptom of something deeper.

You may be able to partially circumvent your printing problem now that we have

  https://git.reviewboard.kde.org/r/130218/

With this patch, if you select rasterized printing, then Okular will render (almost) right into a QPrinter object, rather than converting the file to postscript and rendering that. I'm fairly confident that this will make your stamps appear on the printout.

Revision history for this message
In , Nate-b (nate-b) wrote :

*** Bug 377567 has been marked as a duplicate of this bug. ***

Revision history for this message
In , Haxtibal (haxtibal) wrote :

Just had a look into it, this is what I found. Dependent on users choice, Okular sets the annotation dicts /Name entry to standard names ("Approved", "Confidential", ... see PDF 32000-1:2008, chapter 12.5.6.12 Rubber Stamp Annotations, Table 181), but also to non standard names ("okular", "kde", <path_to_your_custom_image>). Conforming readers implement a representation for the standard names (may look a bit different as in Okular, but at least something reasonable is shown). But of course non standard names like "okular", "kde", <some_local_path> are not understood by other readers. A portable way to have such custom stamps would be to add an /AP entry with the full icon representation into the PDF. Unfortunately, Poppler can't generate appearance streams for stamps yet. Note that Poppler #23108 is a bit different, it is about hardcoded default appearance streams, not about generating ones for your custom image.

There's another special thing about stamps. Contrary to most other annotation types, stamps are not rendered by Poppler, but by Okular itself. Okular first looks for /Name in ui/data/stamps.svg (contains "approved", "confidential", "departmental"...). If there's no match, it uses KIconLoader lookup an icon. Not rendering with Poppler probably introduces special handling when printing, haven't checked that yet.

I guess an ideal solution for this bug will require aid from Poppler side (maybe a new API to store raster and vector graphics as appearance stream into the PDF, and some predefined appearance streams for the standard names so that stamps without /AP can be rendered).

Revision history for this message
In , Oliver-sander (oliver-sander) wrote :

Should we split this into a set of separate smaller issues, then? E.g., issues with stamps with standard appearances seem to be quite independent from custom stamps.

Revision history for this message
In , Haxtibal (haxtibal) wrote :

How about this breakdown? Boils down to the same root causes a bit...

Okular #1: Stamp Annotations are not printable
(I can confirm this, for both standard and non-standard icon names)
Root cause: PDF print requests are forwarded from Okular to the Poppler generator (see PDFGenerator::print). But Poppler can't render stamps for two reasons:
-Poppler doesn't implement default appearance streams for standard icon names, so it can't render stamps like "Draft" and "Approved". See [1], "I leave the bug open becuase there are still annotations missing default appearance streams (Free Text annots, rubber stamp, ...)".
-Poppler can't generate/render/save an appearance streams for your custom *.svg, *.png, whatever image yet. Displaying them in Okular on screen only works because Okular renders stamps on its own.

Okular #2: Custom stamp annotations (i.e. non standard icon name) are not visible in other readers
Root cause: If stamp is created by Okular+Poppler, no appearance stream is generated/embedded into the PDF. Other readers should implement a default appearance stream for standard icon names, but for non-standard icon names like "kde", "okular", <path_to_local_file> they can't know what to display. Maybe of interest is this pdf.js (firefox) issue [2]: "Most annotations, even unsupported ones, render just fine in PDF.js because of their appearance stream. However, there exist PDF files with annotations that do not have an appearance stream (even though this is a deprecated practice). In the latter case, PDF.js displays nothing.". There have been some thoughts about handling of appearance streams in Poppler [3], but I don't know the current state and desire there.

[1] https://bugs.freedesktop.org/show_bug.cgi?id=23108#c21
[2] https://github.com/mozilla/pdf.js/issues/6810
[3] https://lists.freedesktop.org/archives/poppler/2009-June/004789.html

Revision history for this message
In , Oliver-sander (oliver-sander) wrote :

Is it too late to turn this into yet another gsoc project?

Revision history for this message
In , Albert Astals Cid (aacid) wrote :

(In reply to Oliver Sander from comment #9)
> Is it too late to turn this into yet another gsoc project?

No, the student application deadline is March 27 https://developers.google.com/open-source/gsoc/timeline

Revision history for this message
In , Oliver-sander (oliver-sander) wrote :

Can somebody please post this on the gsoc project list for me? Thanks!

Project: Improve custom stamp annotation handling

Brief explanation: Okular does display stamp annotations, but the support
is somewhat incomplete. This particularly shows when trying to use stamp
annotations with a custom image. For example, such annotations can be
added in Okular, but they cannot be saved to the pdf file in a way
that any other pdf viewer can read. Also, they will not appear on print-outs.

The underlying reason for this is that Okular renders these stamps itself,
rather than relying on the poppler library, which does all other pdf
rendering. Goal of this project is therefore to teach poppler how to
render stamp annotations, and then make Okular use that new functionality.
More details can be found in the bug report [0].

[0] https://bugs.kde.org/show_bug.cgi?id=383651

Expected results: Poppler should render stamp annotations. Annotations
should be printable from Okular. Custom stamps inserted via the Okular GUI
should be visible in other pdf readers.

Knowledge prerequisite: C++, and a bit about the pdf format.

Revision history for this message
In , Albert Astals Cid (aacid) wrote :

You should really add it to the wiki for yourself, *personally* i've like 100 unread emails, half of which are people waiting for me to review their code, so I really don't have time to add it myself.

Revision history for this message
In , Oliver-sander (oliver-sander) wrote :

Albert, you are not the only busy person on this planet. Last time I tried I couldn't log in to the wiki page. Now it works all of a sudden. What do I know...

Revision history for this message
In , Nate-b (nate-b) wrote :
Revision history for this message
In , Nate-b (nate-b) wrote :

*** Bug 394620 has been marked as a duplicate of this bug. ***

Revision history for this message
In , Burton+bugs-kde-org (burton+bugs-kde-org) wrote :

This bug is related to the issue of default annotation appearance handling.

It isn't just "image stamp" handling.

There are two bugs here.

https://bugs.freedesktop.org/show_bug.cgi?id=23108

https://bugs.freedesktop.org/show_bug.cgi?id=106635

My bug was here but marked as a duplicate of this bug.

https://bugs.kde.org/show_bug.cgi?id=394620

I'd like to propose that we create a tracking bug or merge this with a larger bug regarding annotation appearance.

What I'm worried about is that if it goes to GSoC or someone else handles it they won't resolve the larger issue of highlight annotations, note annotations, etc. All these are VERY important to resolve now.

Right now Okular generates PDFs that aren't usable by other clients and this is dangerous.

Revision history for this message
In , Nate-b (nate-b) wrote :

I agree that this is important and fixing it should be a high priority.

We have a number of other bugs relating to this subject:
- https://bugs.kde.org/show_bug.cgi?id=275371
- https://bugs.kde.org/show_bug.cgi?id=378186
- https://bugs.kde.org/show_bug.cgi?id=390293

Are any of those more appropriate?

There's no real benefit to having a master "annotations don't work in other PDF readers" bug. Better to track the individual issues with individual bugs, IMHO.

Revision history for this message
In , Burton+bugs-kde-org (burton+bugs-kde-org) wrote :

Thanks Nate.

I think one main concern I have is for other users of Okular using annotations.

Some of these bugs are rather old and I suspect the issue of highlights and notes not being visible impacts Okular from years ago.

If we can't fix these in the next release, I'd like to propose that the annotation functionality be disabled.

The reason is that if a user installs Okular, then migrates away from Okular (say they go from Ubuntu to MacOS/Windows) then they lose all their annotations essentially.

This is a data loss issue.

If the user can't see on the page where his annotations were, in other PDF readers, they're essentially gone.

Revision history for this message
In , Nate-b (nate-b) wrote :

That has been discussed before and rejected, since that would represent a loss of functionality--a great deal of which *does* work in other viewers, and all of which works in Okular itself.

Let's focus on fixing the bugs rather than getting into the weeds with a bruising argument about whether the feature is ready for prime time yet. It's clearly not, but turning it off until it is isn't an option.

Are you a programmer? Are you able to review any of the poppler patches floating around?

Revision history for this message
In , Burton+bugs-kde-org (burton+bugs-kde-org) wrote :

I am a programmer. I'm able to review them and will get to them. I'm curious if this is the best solution. If it means just revising the poppler patches and that's the best path for 80% of the functionality then I'm good to go.

Also, another solution might just be a warning to the user the the annotations MAY not be viewable in all PDF editors and that we don't control those. Then show them the warning once.

My concern is that some of these bugs seem like they won't be readily resolved but a warning might be the best approach here.

Data loss can really hit you

Revision history for this message
In , Nate-b (nate-b) wrote :

I know, and I don't necessarily disagree with you. But as a fellow programmer someone who's watched this project for a while and advocated for the same things, let me tell you that disabling the feature or showing a warning is not going to be accepted. It is what it is, sorry.

Let's focus on fixing the bugs. Where do you think we should start?

Revision history for this message
In , Burton+bugs-kde-org (burton+bugs-kde-org) wrote :

Do you have a thought on the poppler default annotation appearance issue?

That's probably the greatest bang for the buck. If that fixes 80% of the problems then we should focus a patch there.

I've asked for commentary on their bugs.

Revision history for this message
In , Nate-b (nate-b) wrote :

I agree that we should focus on Poppler. If I'm reading https://bugs.freedesktop.org/show_bug.cgi?id=23108#c21 correctly, patches were merged that handled some annotation types, but not all of them. We should probably focus on the remaining annotation types that still need default Poppler appearance streams.

Revision history for this message
In , Burton+bugs-kde-org (burton+bugs-kde-org) wrote :

I'll probably have to look and see if we can confirm that this is the problem.

maybe a test app that generates all the annotation types ...

I'm working on a separate project that can export the annotations as JSON from PDF.js and make them machine readable.

I might be able to include the colors there so that we could have a unit test or some way to verify that they work.

poppler might already have some options for this.

Revision history for this message
In , Dimitrios-tanis (dimitrios-tanis) wrote :

Is there any update on this?
I agree with Kevin that data loss can be a buzz but when advocating for the bells and whistles of linux and KDE as a pro business environment, sending an annotated pdf that others can't view is even worst (I've just been hit by that).

It's been quite a few years since I've coded in C++ and Qt so I can't be of use in coding but I can help with testing if needed.

Revision history for this message
In , O-valentin (o-valentin) wrote :

Hello, is there some news about this bug? It is still happening in version 1.6.3. Thanks in advance!

Revision history for this message
In , Haxtibal (haxtibal) wrote :

(In reply to Valentin Melot from comment #26)
> Hello, is there some news about this bug? It is still happening in version
> 1.6.3. Thanks in advance!
Afaikt there's no news, sorry.

@Nate, Kevin: You seem to focus on default appearances. I don't think they would fix this bug, can you explain how?

For my understanding we need to extend poppler so that it can generate and save appearance streams into the PDF file in a consistent way:
- either generate PDF operations from user image to draw image as paths
- or rasterize / convert user image to a sampled PDF image
- create AP structure, enter the generated image stream, enter additional resources
- update all resource references throughout the document
- save new and dirty objects

That all has not much to do with default appearances. Even if poppler had hard coded default appearances for stamps, non-Okular readers would not benefit from them unless the appearance is saved into the file, which is currently not possible and which is the toughest part to implement. And, "default" is quite the opposite to "custom" (as in the bug title).

Revision history for this message
In , Simone Gaiarin (simgunz) wrote :

Git commit f15e8568a5330b6795b1dd5287a6a1530dc60476 by Simone Gaiarin.
Committed on 25/07/2019 at 18:09.
Pushed by gaiarin into branch 'master'.

General improvements to stamp annotation

Summary:
Configuration:
- Add push button to select custom stamp image
- Check if loaded image is usable as stamp or throw error
- Keep image proportions in previewer
- Move previewer below the combobox to display larger preview

Annotation tool:
- Keep stamp image proportion in annotation preview (while left mouse button is down)
- Adding the annotation with one-click (without holding the left mouse button and dragging) adds the stamp with original proportions
Related: bug 370381, bug 383652
FIXED-IN: 1.9.0

Closes T8074

TODO:
- [ ] Check if filters in file chooser make sense / propose better alternative
- [x] Update doc ( @yurchor will do it after we merge this)

Test Plan:
>From stamp annotation configuration dialog:
- Show a warning regarding limitations of the feature's current implementation
- Click push button next to combo box opens a file selector
- Selecting a corrupted image file should throw an error
- Selecting a good image file shows the preview of the image
- Select a horizontal image shows a large clear preview
- Select a vertical image file shows a smaller preview without messing up the visual of the config dialog
- Input a valid icon name in the combobox and the preview of the icon is shown

>From page view, select the stamp annotation with horizontal image file (not squared):
- Click and hold. The preview maintains proportions
- Single click. The stamp image in the pdf maintains proportions and has the same size of the click and hold preview.
- Add an annotation of the Okular custom stamps (internal SVG so treated slightly differently) do not create problems

Reviewers: #okular, ngraham

Reviewed By: ngraham

Subscribers: pino, aacid, yurchor, ngraham, okular-devel

Tags: #okular

Maniphest Tasks: T8074

Differential Revision: https://phabricator.kde.org/D22064

M +62 -9 ui/annotationwidgets.cpp
M +7 -1 ui/annotationwidgets.h
M +25 -9 ui/guiutils.cpp
M +8 -1 ui/guiutils.h
M +1 -1 ui/pagepainter.cpp
M +4 -4 ui/pageviewannotator.cpp

https://invent.kde.org/kde/okular/commit/f15e8568a5330b6795b1dd5287a6a1530dc60476

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in okular (Ubuntu):
status: New → Confirmed
Revision history for this message
In , Denny (denny-seniazi) wrote :

SUMMARY
I need a pdf file including my signature in a non-editable format.I have created a respective custom annotation. As there is no option to "flatten" annotations, I thought printing to pdf would be a suitable workaround. This workaround works for other annotations, but not for stamps (even for non-custom).

STEPS TO REPRODUCE
1. Open a pdf file
2. add a stamp annotation,
3. print to pdf (tick print annotations).

OBSERVED RESULT
A pdf without stamp annotations

EXPECTED RESULT
A pdf with non-editable ("flattened") stamp annotations

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Kernel 5.3.0/ KDE plasma 5.16.5
(available in About System)
KDE Plasma Version: 5.16.5
KDE Frameworks Version: 5.62.0
Qt Version: 5.12.4

ADDITIONAL INFORMATION

Revision history for this message
Denny (denny-seniazi) wrote :
Changed in okular:
importance: Unknown → Medium
status: Unknown → New
Revision history for this message
In , Albert Astals Cid (aacid) wrote :

Yeah, basically "stamps don't really work", they just work if you never leave Okular user interface.

There's various bugs about it

Revision history for this message
In , Nate-b (nate-b) wrote :

*** This bug has been marked as a duplicate of bug 383651 ***

Revision history for this message
In , Nate-b (nate-b) wrote :

*** Bug 416252 has been marked as a duplicate of this bug. ***

Changed in okular:
status: New → Invalid
Changed in okular:
status: Invalid → Unknown
Revision history for this message
In , Albert Astals Cid (aacid) wrote :

*** Bug 435904 has been marked as a duplicate of this bug. ***

Revision history for this message
In , Nate-b (nate-b) wrote :

*** Bug 447086 has been marked as a duplicate of this bug. ***

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.