Does not support IDNs with SMTPUTF8.

[Lolalolo]

dkimpy does not sign E-Mails send over SMTPUTF8 while there are in their UTF8 Form. It does not try to convert it to punycode and if i try to provide an entry in the key-table in utf8, it just does not sign it and crashes.
sign_dkim: 'ascii' codec can't encode characters in position 0-4: ordinal not in range(128)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/Milter/__init__.py", line 772, in <lambda>
  milter.set_eom_callback(lambda ctx: ctx.getpriv().eom())
File "/usr/lib/python3/dist-packages/dkimpy_milter/__init__.py", line 198, in eom
  self.sign_dkim(txt)
ilter[169754]: File "/usr/lib/python3/dist-packages/dkimpy_milter/__init__.py", line 319, in sign_dkim
  h = d.sign(codecs.encode(self.selectorRSA, 'ascii'), codecs.encode(self.fdomain, 'ascii'),
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-4: ordinal not in range(128)

[Scott Kitterman (kitterman)]

I agree this is something that should work, but I'll need to research the RFC updates on IDN for DKIM to see what the best way to support this is. I saw the original bug you filed against dkimpy. I'm not sure that was wrong, but I can reassign it if needed. d.sign here is a dkim.DKIM object.

[Lolalolo (lfsjfsjdfs-deactivatedaccount)]

But the Error is not in dkimpy, the error is in
`codecs.encode(self.fdomain, 'ascii')`. I think the best option to
implement it is to first remember in which form the sender#s address is
a label or u label, and then downgrade that to a label to find the keys
and the signing table entry. The signature should then use the same form
in the d= tag, as the sender's address used.

Best Regards

Jannes Althoff

On 14.01.22 06:21, Scott Kitterman wrote:
> I agree this is something that should work, but I'll need to research
> the RFC updates on IDN for DKIM to see what the best way to support this
> is. I saw the original bug you filed against dkimpy. I'm not sure that
> was wrong, but I can reassign it if needed. d.sign here is a dkim.DKIM
> object.
>
>
> ** Changed in: dkimpy-milter
> Status: New => Confirmed
>

[Scott Kitterman (kitterman)]

They probably both need work. Here's an excerpt from https://www.rfc-editor.org/rfc/rfc8616.txt which added IDN support to DKIM:

   Section 3.5 of [RFC6376] states that IDNs in the d=, i=, and s= tags
   of a DKIM-Signature header field MUST be encoded as A-labels. This
   rule is relaxed only for internationalized message header fields
   [RFC6532], so IDNs SHOULD be represented as U-labels. This provides
   improved consistency with other header fields. (A-labels remain
   valid to allow a transition from older software.) The set of
   allowable characters in the local part of an i= tag is extended in
   the same fashion as local parts of email addresses as described in
   Section 3.2 of [RFC6532]. When computing or verifying the hash in a
   DKIM signature as described in Section 3.7 of [RFC6376], the hash
   MUST use the domain name in the format it occurs in the header field.

The pymilter bindings that we use to connect to libmilter were just updated recently to support non-ascii, but I haven't had time to work on it yet. That was a prerequisite to any of this working the way RFC 8616 wants it to work.

If you can manually convert the domain name to a A-label and use that in your configuration it might work. I'd be interested to know. In any case, IDN support is the number one feature on my list for both dkimpy and dkimpy-milter when I can get to it.

[Lolalolo (lfsjfsjdfs-deactivatedaccount)]

I got signatures to work by just replacing ascii in all locations with
UTF-8 in dkimpy-milter.

But I do not actually know if the signature is valid, because GMail has
an error in its IDN U to A-Label Algorithm it treats both sigmas the
sigma used at the end (U+03C3) and the sigma used in a middle of a
sentence (U+03C2) the same. And Outlook just doesn't find my DKIM, DMARC
and SPF Records. There is somewhere there an error in there software.

On 14.01.22 06:45, Scott Kitterman wrote:
> They probably both need work. Here's an excerpt from https://www.rfc-
> editor.org/rfc/rfc8616.txt which added IDN support to DKIM:
>
> Section 3.5 of [RFC6376] states that IDNs in the d=, i=, and s= tags
> of a DKIM-Signature header field MUST be encoded as A-labels. This
> rule is relaxed only for internationalized message header fields
> [RFC6532], so IDNs SHOULD be represented as U-labels. This provides
> improved consistency with other header fields. (A-labels remain
> valid to allow a transition from older software.) The set of
> allowable characters in the local part of an i= tag is extended in
> the same fashion as local parts of email addresses as described in
> Section 3.2 of [RFC6532]. When computing or verifying the hash in a
> DKIM signature as described in Section 3.7 of [RFC6376], the hash
> MUST use the domain name in the format it occurs in the header field.
>
> The pymilter bindings that we use to connect to libmilter were just
> updated recently to support non-ascii, but I haven't had time to work on
> it yet. That was a prerequisite to any of this working the way RFC 8616
> wants it to work.
>
> If you can manually convert the domain name to a A-label and use that in
> your configuration it might work. I'd be interested to know. In any
> case, IDN support is the number one feature on my list for both dkimpy
> and dkimpy-milter when I can get to it.
>

[Lolalolo (lfsjfsjdfs-deactivatedaccount)]

This is an example what i got.

X-Original-To: <email address hidden>
Delivered-To: <email address hidden>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
 d=κλαρα-σωλις.ευ; i=@κλαρα-σωλις.ευ;
 q=dns/txt; s=2022; t=1642119943; h=message-id : date : mime-version :
 to : from : subject : content-type : content-transfer-encoding : from;
 bh=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=;
 b=KR7AGlkgxyOX3jR62O4s+5dehey/2y5KMBGLVuwUclK7SQlD57TlbGlAgTmSqgIkTaiAJ
 xlwuFOzbwVcjX6VpvuWsEghDItWhA7xj7yW9AuGVXsHClRTyKVMK233q1cpHQ/MV/qfBwO2
 U6hajK1VT4f6vscROU3BfY1DKh12p7I60PrHCZ0PTA0pwBf0gkT3hdWFED8emrp9kKLFIda
 B33LmFGxSjPYhJJcJeTv1bHE9sXIvQqhUkqoOpIv6AS8qHXKW9+8Om+Ki9pWb06RqevkgKr
 Zj4StXlnJZUbiAJ70ltg/mABtYD6wiU+PmVBDoZZqJheoRyMH4q90L0N0FOA==
Received: from [IPV6:2003:ec:2f12:3c00:932d:77bf:9e20:a6bc] (p200300ec2f123c00932d77bf9e20a6bc.dip0.t-ipconnect.de [IPv6:2003:ec:2f12:3c00:932d:77bf:9e20:a6bc])
 (Authenticated sender: <email address hidden>)
 by mail.jannes-althoff.de (Postfix) with UTF8SMTPSA id 64B8180D2A
 for <email address hidden>; Fri, 14 Jan 2022 00:25:43 +0000 (UTC)
Message-ID: <email address hidden>
Date: Fri, 14 Jan 2022 01:25:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.4.1
Content-Language: en-US
To: <email address hidden>
From: Jannes Althoff <jannes.althoff@κλαρα-σωλις.ευ>
Subject: fdfd
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

[Scott Kitterman (kitterman)]

Thanks. I'm surprised it worked that well.

[Lolalolo (lfsjfsjdfs-deactivatedaccount)]

To clear any misunderstanding, I mean I changed the Source Code of
dkimpy-milter and changed every mention of ascii to utf8. The
dkimpy-milter library is filled with instructions to decode with ascii.
But because I changed the library I don't really trust the validity. I
checked it for non UTF8 E-Mails and Domains and this worked. But I have
currently no way to check it for UTF8 E-Mails.

On 14.01.22 14:44, Scott Kitterman wrote:
> Thanks. I'm surprised it worked that well.
>

[Scott Kitterman (kitterman)]

Thanks.