Current 14.04 bitlbee build using broken OTR (fixed in nightlies)

[William Budington]

The current version of bitlbee, bitlbee-common, and bitlbee-plugin-otr in 14.04 is 3.2.1+otr4-1. In this version, there are numerous problems in the OTR negotiation phase. These problems have been fixed in the nightlies for bitlbee. Relevant tickets are:

http://bugs.bitlbee.org/bitlbee/ticket/1109
http://bugs.bitlbee.org/bitlbee/ticket/1110

[Impact]

- BitlBee crashes when sending messages to contacts with no previous otr context (offline contacts, or contacts that weren't talked to before)

- BitlBee silently drops all unmodified messages when opportunistic otr is enabled, which is the default mode

- Receiving a /me from a remote contact crashes BitlBee.

[Test Case]

- Start bitlbee daemon / service

- Start irc client, for example irssi

- Connect to bitlbee with "/connect localhost"

- Switch to the autojoined "&bitlbee" channel

- Type "add jabber username@domain password" with appropriate login details (a jabber account in any server will do, gmail also works)

- Type "account jabber on" to connect.

And then either:

1.1 Find an offline contact (type "blist all" in &bitlbee to show them)
1.2 Open a PM window with that contact ("/query contact_name")
1.3 Send two messages to that contact
1.4 Watch it crash (will appear as a disconnection from the server in the irc client)

2.1 Do "set otr_policy manual"
2.2 Open a PM window with an online contact.
2.2 Send at least one message.
3.3 Ask the contact (using some other communication method) if they received anything.

3.1 Keep otr_policy in the default value (or reset it with "set otr_policy opportunistic")
3.2 Send at least two messages to an online contact
3.3 Ask the contact (using some other communication method) if both of them were received.

4.1 Ask the contact (using some other communication method) to send a "/me"-equivalent action
4.2 Watch it crash

[Regression Potential]

- Highly unlikely if only otr.c patches are backported - otr.c is already very very broken.

- Possible but still unlikely if a full upload of 3.2.2-2 is done (the current stable release, which is mostly 3.2.1+otr4-1 with fixes)

- Highly unlikely if the OTR plugin is removed (see below)

- These have been tested and worked perfectly for all the users that complained about it so far in our channel (roughly 50 of them), who were told to switch to the nightly apt repo, and this turned out to be more stable.

[Other Info]

- Backporting individual otr-related patches is one way to solve the issue, although not my preferred one.

- A full upload of 3.2.2-2 would be the easiest way to solve it. It's stable, and the otr patches in 3.2.1+otr4-1 are the biggest part of it.

- Removing the OTR plugin package will also get rid of the issues, probably the safest option, but with a functionality loss. This will require changing the version string to not say "otr4", as this is misleading - without changing that, it will not be an effective way to solve our tech support headache in #bitlbee

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[dx (dx)]

This is fixed in the 3.2.2 release, which introduces proper OTR support.

To summarize the issues found and fixed:
- crashing when sending messages to contacts with no previous otr context
- silently dropping all messages when opportunistic otr is enabled, which is the default mode
- receiving a /me from a remote contact crashes bitlbee.

In other words, it's unusable. Any chances the 14.04 package can be updated to this version?

[dx (dx)]

3.2.2-2 is in vivid now, arrived a bit too late for utopic :(

I'll be looking into what's needed to SRU this

[dx (dx)]

So apparently I can change the status of this issue to "fix released" (in vivid)

[dx (dx)]

Wait. Dammit. Wrong status. Can't set it back.

[Jelmer Vernooij (jelmer)]

vivid has 3.2.2-2, isn't that recent enough?

[dx (dx)]

Oh thanks jelmer for appearing. #ubuntu-bugs didn't reply ;_;

So "fix released" is okay even if it's just the development version, and the ticket is about trusty and utopic?

[dx (dx)]

Updated description to fit the SRU template, and I just noticed that each description change shoots an email. Sorry for that!

[dx (dx)]

02:07 < ScottK> If you know how to fix it, attach the patch to a bug, explain what's going on and subscribe the ubuntu-sponsors team to the bug.
02:08 < ScottK> All the distro specific specific bureaucracy they should be able to handle.
02:10 < dx> there are several bugs (i think three of them, two critical, one not so much), is it okay if i submit a single patch with the three of them?
02:12 < Logan> yes

So here it is. It was four bugs, three critical (message loss, crash, crash), one not so much (otr connect is a no-op)

Log message:

    Backport bitlbee-plugin-otr bug fixes to 3.2.1+otr4-1

    Includes the following git commits, slightly adapted to apply cleanly
    after removing a commit that was a failed fix attempt.

    367ea3c: work around libotr 4 not sending outgoing plaintext messages
    329f9fe: use OTRL_INSTAG_BEST instead of _RECENT to work around a segfault in libotr
    74c9e7f: fix a segfault when otr-coloring /me messages
    820a2a7: fix 'otr connect' command

    Relevant bugs:

    - https://bugs.bitlbee.org/bitlbee/ticket/1109
    - https://bugs.bitlbee.org/bitlbee/ticket/1110
    - https://bugs.otr.im/issues/16 (not fixed, but workarounded here)

[dx (dx)]

Oh I should probably expand on the "what's going on": this is still very broken in trusty (even though it says fix released - that's vivid) and we still regularly get users complaining about it every few months. I don't want to keep dealing with the issue for the next few years. Please.

[Timo Jyrinki (timo-jyrinki)]

Thanks for caring about the users, Dx!

I verified the backported commits seem sane and tested build + briefly started it. I've uploaded it to the queue https://launchpad.net/ubuntu/trusty/+queue?queue_state=1&queue_text= - when it gets approved from there, this bug gets updated with instructions on how to do the Stable Release Update verification. Verification should be done by someone who's a daily user of bitlbee (I'm not myself).

[Chris J Arges (arges)]

Hello William, or anyone else affected,

Accepted bitlbee into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/bitlbee/3.2.1+otr4-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[dx (dx)]

3.2.1+otr4-1ubuntu0.1 still segfaults.

A disasm at the point of the segfault, near the end of otr_filter_msg_out(), shows that it didn't change. The last few calls are still otrl_message_sending() and g_free(), as opposed to g_strdup(), otrl_message_free() and irc_usernotice()

Is this something related to the 1.0 source format?

[Timo Jyrinki (timo-jyrinki)]

@Dx: Argh, so it is. And the packaging is also a bit archaic in other ways, so I would just apply the patch manually similar to the config patch. Could you please test the 3.2.1+otr4-1ubuntu0.2~test2 from ppa:timo-jyrinki/quantal-compiz-unity-testing ? (don't mind the PPA name, my normal PPA is full so I couldn't use it). I don't want to do another upload if the actual bug is not fixed.

[dx (dx)]

Nope! Same thing!

Here, have a command to check if the last part of that function changed:

# gdb -q -ex 'disas otr_filter_msg_out+240,+50' -ex q /usr/lib/bitlbee/otr.so

Reading symbols from /usr/lib/bitlbee/otr.so...(no debugging symbols found)...done.
Dump of assembler code from 0x479e to 0x47d0:
   0x000000000000479e <otr_filter_msg_out+240>: xor $0x207c7d,%eax
   0x00000000000047a3 <otr_filter_msg_out+245>: mov %rax,%rdi
   0x00000000000047a6 <otr_filter_msg_out+248>: callq 0x3890 <otrl_message_sending@plt>
   0x00000000000047ab <otr_filter_msg_out+253>: mov %eax,-0x2c(%rbp)
   0x00000000000047ae <otr_filter_msg_out+256>: mov -0x18(%rbp),%rax
   0x00000000000047b2 <otr_filter_msg_out+260>: cmp -0x40(%rbp),%rax
   0x00000000000047b6 <otr_filter_msg_out+264>: je 0x47c4 <otr_filter_msg_out+278>
   0x00000000000047b8 <otr_filter_msg_out+266>: mov -0x18(%rbp),%rax
   0x00000000000047bc <otr_filter_msg_out+270>: mov %rax,%rdi
   0x00000000000047bf <otr_filter_msg_out+273>: callq 0x3220 <g_free@plt>
   0x00000000000047c4 <otr_filter_msg_out+278>: mov $0x0,%eax
   0x00000000000047c9 <otr_filter_msg_out+283>: leaveq
   0x00000000000047ca <otr_filter_msg_out+284>: retq
   0x00000000000047cb: push %rbp
   0x00000000000047cc: mov %rsp,%rbp
   0x00000000000047cf: sub $0x20,%rsp
End of assembler dump.

[Timo Jyrinki (timo-jyrinki)]

Third time the charm! If you look at the build log, the patch was applied _after_ the compilation. I now moved it to the beginning of the build phase as it should be.

So, please test ~test4 and maybe we soon have SRU:able fix.

The packaging would use an overall renewal but for SRU the fewer lines changed the better.

[dx (dx)]

Huge success! All four test cases in the ticket description are now passing.

[Timo Jyrinki (timo-jyrinki)]

Thanks! I uploaded 0.2 now to the queue again. I should have asked your for testing a "~test" version the first time too, to catch this without the loop to the distro and back.

[Chris J Arges (arges)]

Would it be possible to actually fix the package so it properly handles quilt patches? Seems like a big hack to have to introduce patches manually in rules like this especially if its already using debhelper.

[dx (dx)]

There's this pending pull request, which i have no idea what to do with: https://github.com/bitlbee/bitlbee/pull/43

It changes the format from "1.0" to "3.0 (quilt)" and by doing so it breaks the build (it's got a red X from the travis-ci build).

Specifically, dpkg-buildpackage throws:

>dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../bitlbee_3.4.1.orig.tar.{bz2,gz,lzma,xz}

I don't know enough about deb packaging to fix it so i left it on hold. Some help would definitely be welcome.

[Jelmer Vernooij (jelmer)]

On Wed, Oct 28, 2015 at 09:06:10PM -0000, Dx wrote:
> There's this pending pull request, which i have no idea what to do with:
> https://github.com/bitlbee/bitlbee/pull/43
>
> It changes the format from "1.0" to "3.0 (quilt)" and by doing so it
> breaks the build (it's got a red X from the travis-ci build).
>
> Specifically, dpkg-buildpackage throws:
>
> >dpkg-source: error: can't build with source format '3.0 (quilt)': no
> upstream tarball found at ../bitlbee_3.4.1.orig.tar.{bz2,gz,lzma,xz}
>
> I don't know enough about deb packaging to fix it so i left it on hold.
> Some help would definitely be welcome.

You need to have an upstream tarball (named e.g. bitlbee_3.4.1.orig.tar.gz)
available in the parent directory.

The 1.0 format falls back to a "native" package if it can't find the upstream
tarball, which is probably not what you want.

Cheers,

Jelmer

[dx (dx)]

>You need to have an upstream tarball (named e.g. bitlbee_3.4.1.orig.tar.gz) available in the parent directory.
>The 1.0 format falls back to a "native" package if it can't find the upstream tarball, which is probably not what you want.

Uh, well, I don't know what to do with this information. All I know is that the 3.0 format doesn't work in travis-ci and 1.0 does. The actions needed to upgrade to the 3.0 format without breaking anything are still a mystery to me.

Either way it's not very relevant to this ticket, that discussion can continue in https://github.com/bitlbee/bitlbee/pull/43

Unless upgrading the source format is required for this SRU to happen. Is it?

[Timo Jyrinki (timo-jyrinki)]

I think with the debian/rules being the mess it is, the current way of applying a patch manually there (another one, there's an existing one already) would be fine for SRU. I agree the packaging should be improved, but the debian/rules should be rewritten from scratch which is not appropriate for SRU.

[Chris J Arges (arges)]

I agree, I wasn't aware of the magnitude of changes necessary. Would be nice to fix it properly in the future.

[Chris J Arges (arges)]

Hello William, or anyone else affected,

Accepted bitlbee into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/bitlbee/3.2.1+otr4-1ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[dx (dx)]

Got 3.2.1+otr4-1ubuntu0.2, all four tests passing. Everything else seems to be working normally too.

...Is this enough for a SRU verification? Do I just go ahead and change tags now?

[Timo Jyrinki (timo-jyrinki)]

@Dx: That's enough, I can mark the tag for you.

[Launchpad Janitor (janitor)]

This bug was fixed in the package bitlbee - 3.2.1+otr4-1ubuntu0.2

---------------
bitlbee (3.2.1+otr4-1ubuntu0.2) trusty; urgency=medium

  * debian/rules: apply the patch manually (LP: #1315550)
  * debian/patches/series: remove, not needed

 -- Timo Jyrinki <email address hidden> Thu, 22 Oct 2015 09:11:15 +0300

[Chris J Arges (arges)]

The verification of the Stable Release Update for bitlbee has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[dx (dx)]

Thanks to everyone who helped <3

Really happy to finally be able to consider this completely solved :D