SendEmail A/T reactor broken for recent version of Encode::MIME::Header

[Galen Charlton]

The approach introduced in commit d88e3624ea4 (see bug 1031335) to MIME-encode headers does not work when using recent versions of Encode::MIME::Header. Specifically, Encode::MIME::Header 2.22 or later, which is part of Encode 2.83 or later, adopts a much stricter view of its task and does not work for structured header fields like To or From.

In particular, older versions of Encode::MIME::Header would yield results like this:

From: stuff <email address hidden>

while newer version give you something like

From: =?UTF-8?B?c3R1ZmYgPGZvb0BiYXIuY29t?=

Email::Send chokes on the latter when it tries to determine the sender address to use for the SMTP transaction.

Since the maintainers of Encode::MIME::Header are unwilling (see https://github.com/dankogai/p5-encode/issues/105) to make it handle structured headers in a more "bug-compatible" fashion, we'll need to do something else. Email::MIME or Email::Stuffer both look promising.

Marking this high importance because it's evident that there's at least one case where an innocent upgrade of Encode results in breaking outgoing A/T email.

Evergreen 3.1+

[Jason Stephenson (jstephenson)]

I want to point out related bug 1466502, and bug 1362260 which took the approach of installing Email::Send from CPAN.

Perhaps an alternate solution is to finally switch to Email::Sender?

[Galen Charlton (gmc)]

I think the issues are orthogonal, as at first glance Email::Sender still needs something like Email::MIME to generate a correctly formatted (and escaped) email to send.

[Jason Stephenson (jstephenson)]

OK, Galen, I see you point after reading through the Github issue.

I want to add that it looks like this issue will affect Ubuntu 18.04 support. I'm willing to work on this at the Hack-A-Way next week. My Ubuntu 18.04 VM has Encode version 2.88 and Perl 5.26.

[Galen Charlton (gmc)]

Thanks! It occurs to me that dealing with Email::Sender at or around the same would be a good idea; if we end up adding a dependency on Email::MIME or Email::Stuffer, we may as well stack up other dependency changes.

[Jason Stephenson (jstephenson)]

I have pushed a branch to the working repository that resolves this issue in my testing (see the commit message for a test procedure).

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dyrcona/lp1801163-email-mime

Since this branch adds prerequisites, I hesitate to say this should be backported to 3.2 or earlier releases.

[Galen Charlton (gmc)]

I've tested and pushed a signoff to user/gmcharlt/lp1801163_email_mime_signoff.

The signoff branch contains two follow-ups:

[1] Adding the new dependencies to the Buster and Fedora installation. I've not actually tested the Fedora part, though.
[2] Ensure that non-ASCII characters in the headers don't get mangled.

Since without this bugfix SendEmail appears to be completely broken on (at least) Debian Stretch and later and Ubuntu 18.04 and later, I think we'll need to bite the bullet and backport this.

[Jason Boyer (jboyer)]

I've given this one last once-over and all of the silly unicode that I threw in my template ended up in my inbox. Pushed to master, rel_3_4, and rel_3_3.

Thanks Jason and Galen.