Upgrade Rust to 1.67.1

[Zixing Liu]

Upgrade Rust to 1.67.1

[Zixing Liu (liushuyu-011)]

Preliminary packages are available in https://launchpad.net/~liushuyu-011/+archive/ubuntu/rust-updates-1.67/+packages

[Zixing Liu (liushuyu-011)]

Updated and fixed packages are available in https://launchpad.net/~liushuyu-011/+archive/ubuntu/rust-updates-1.67/+packages

[Zixing Liu (liushuyu-011)]

Notable packaging changes since Rust 1.66.1:
- Disable SSE2-related stuff on i386 (since our baseline is not i686 w/ SSE2)

You can find other changes documented in the changelog of the source package.

[Simon Chopin (schopin)]

Hi Zixing,

I'm assuming you're using https://git.launchpad.net/~canonical-foundations/ubuntu/+source/rustc/log/?h=merge-1.67 for your work. Please don't forget to mention this when asking for sponsorship :)

Could you please use DEP-3 headers on any new patch? https://dep-team.pages.debian.net/deps/dep3/

This makes it easier to reason about patches, share them with other distros, etc...

[Zixing Liu (liushuyu-011)]

Hi Simon,

Yes, the changes are done on https://git.launchpad.net/~canonical-foundations/ubuntu/+source/rustc/log/?h=merge-1.67.

I have added the DEP-3 headers to the new patch introduced for fixing the i386 build (the only new patch).

[Simon Chopin (schopin)]

Uploaded, thanks!

[Zixing Liu (liushuyu-011)]

I have discovered a critical issue with the upload. Can we retract version 1.67.1 from the archive for now?

The binary in the current archive can not find the standard library due to an upstream change: https://github.com/rust-lang/rust/commit/71a3a48ee52a5cbc3fa3e3e8d322ebeb97ed109f

The change in question was a fix to the Windows version of the Rust compiler, but it seems like it broke our situation where the `multiarch` layout is used in /usr/lib.

This change was not tested in the test suite, so it was not caught during the build. When I installed the binary from the archive, I detected this issue.

Sorry for not catching this early. This will also block the Cargo upload for now due to buildability.

[Zixing Liu (liushuyu-011)]

To conclude, how this happened but flew under the radar (both upstream and us):

- The author of that upstream change did not consider the possibility of breaking Linux distros like Debian (they stated in the PR that "this finds the sysroot using Rust driver DLLs."
- "Modern" Linux distros did not encounter this issue because they use non-multiarch layout.
- Some other Linux distros discovered the issue but stayed silent and worked around it using their own mechanisms.
- I did not discover this issue because the new Cargo 0.68 was built using Rust 1.66, and the test suite for Rust 1.67 did not include tests for testing directory look-up.

In the future, I will add a basic smoke test for Rust to prevent this kind of issue and catch the problem early.

[Zixing Liu (liushuyu-011)]

I have added a patch to address this issue.

The patch was also forwarded to Rust upstream: https://github.com/rust-lang/rust/pull/108376.

The fixed batch was uploaded to the PPA as version 1.67.1+dfsg0ubuntu1-0ubuntu1.6: https://launchpad.net/~liushuyu-011/+archive/ubuntu/rust-updates-1.67/+packages

The series of changes to fix the issue were added to https://git.launchpad.net/~canonical-foundations/ubuntu/+source/rustc/tree/?h=merge-1.67 with EVR 1.67.1+dfsg0ubuntu1-0ubuntu2.

[Zixing Liu (liushuyu-011)]

Also, test failures in i386 and ppc64el are due to a recent GDB upload in LL.

The Rust test suite has tests that launch GDB to inspect debug information.

GDB threw some internal errors in those tests:

/build/gdb-i4hVub/gdb-13.1/gdb/frame.c:2457: internal-error: inside_main_func: Assertion `block != nullptr' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
----- Backtrace -----
0x5668dc91 ???
0x56a9bd24 ???
0x56a9bfae ???
0x56bd1378 ???
0x567f0b6f ???
0x567618f1 ???
0x56761e24 ???
0x56761f4b ???
0x569e49f4 ???
0x569e593c ???
0x569e9c9d ???
0x569ea9be ???
0x5684d5a6 ???
0x566cddd5 ???
0x5685002c ???
0x5686222a ???
0x5682fb2e ???
0x56880fa9 ???
0x56bd1575 ???
0x56bd183b ???
0x56bd2406 ???
0x56a53c02 ???
0x56a53c84 ???
0x56a541cb ???
0x567c1e58 ???
0x56a5640e ???
0x566d42c6 ???
0x566c4d8d ???
0x568ac8aa ???
0x568ac96d ???
0x568aeb94 ???
0x568af94f ???
0x565e64d9 ???
0xf6e2a8f8 ???
0xf6e2a9bb __libc_start_main
0x565edb1a ???
0xffffffff ???
---------------------

This is a bug, please report it. For instructions, see:
<https://www.gnu.org/software/gdb/bugs/>.

[Łukasz Zemczak (sil2100)]

Removing packages from lunar-proposed:
 rustc 1.67.1+dfsg0ubuntu1-0ubuntu1 in lunar
Comment: Requested by uploader, contains critical issue: https://bugs.launchpad.net/ubuntu/+source/rustc/+bug/2007642/comments/7
Remove [y|N]? y
1 package successfully removed.

Package should now be removed from lunar-proposed.

[Matthias Klose (doko)]

the rustc testsuite fails with 13.1 on i686-linux-gnu and powerpc64le-linux-gnu, apparently worked before with trunk 20230114, both reported on Debian and Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/rustc/+bug/2007642/comments/10
claims:

/build/gdb-i4hVub/gdb-13.1/gdb/frame.c:2457: internal-error: inside_main_func: Assertion `block != nullptr' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1031745

same internal error

[Hector Oron (zumbi)]

Created attachment 14714
test case with readme

Attaching code that reproduces the issue. See README file.

Original source: https://salsa.debian.org/fg/rustc-gdb-1031745

[Hector Oron (zumbi)]

With attached testcase I am able to reproduce the issue using gdb 13.1, but it works using 13.0.91.20230210.

It might be related to the patch that fixed PR/30116.

[Tromey-b (tromey-b)]

Ugh. I'll investigate.

[Hector Oron (zumbi)]

I confirm that reverting patch for PR/30116 makes the rust testsuite work again (for me).

[Tromey-b (tromey-b)]

The crash is in inside_main_function, due to the modifications here:

    commit 9370fd51ebfca8a8acacaecb92c57ee54f4f8382
    gdb: detect main function even when there's no matching msymbol

First, gdb doesn't find the minsym, probably because the msyms
contain:

[99] t 0x90c0 _ZN17rustc_gdb_10317454main17h5b5be7fe16a97225E section .text rustc_gdb_1031745::main::h5b5be7fe16a97225 zko06yobckx336v

... but we're looking for:

(top-gdb) p name
$13 = 0x292e0c0 "rustc_gdb_1031745::main"

The _Z overloading strikes again.

Anyway somehow we find a full symbol for this, but it doesn't
have a block, so the assert fails.

[Tromey-b (tromey-b)]

rustc decides to emit both:

 <2><2f5b>: Abbrev Number: 10 (DW_TAG_subprogram)
...
    <2f6e> DW_AT_name : (indirect string, offset: 0x23a): main

and

 <2><2f95>: Abbrev Number: 2 (DW_TAG_namespace)
    <2f96> DW_AT_name : (indirect string, offset: 0x23a): main

in the same scope.

Then the lookup finds the wrong one, because gdb's symbol tables
are bad and still follow C namespace rules.

[Tromey-b (tromey-b)]

We can avoid the crash pretty easily but stopping the 'bt'
before _start is harder.

[Tromey-b (tromey-b)]

Another thought here is it would probably be better on the whole
to use the main symbol's linkage name; but that too seems like
a larger change.

[Tromey-b (tromey-b)]

https://sourceware.org/pipermail/gdb-patches/2023-February/197370.html

I'll apply to the gdb 13 branch as well when it goes in.

[Hector Oron (zumbi)]

(In reply to Tom Tromey from comment #9)
> https://sourceware.org/pipermail/gdb-patches/2023-February/197370.html

Thanks! That worked with the test case.

[Zixing Liu (liushuyu-011)]

GDB issues fixed in https://launchpad.net/ubuntu/+source/gdb/13.1-2ubuntu2.
`rustc` builds were retried and confirmed to be passed.

[Hector Oron (zumbi)]

Hello Tom,

  The testcase at https://salsa.debian.org/fg/rustc-gdb-1031745 has been update to add one more failing test case scenario described at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1031946

Fabian reports:
I updated the reproducer repo[0], broken and good output look like this:

----8<---- broken
Breakpoint 1 at 0x81dd: file src/main.rs, line 88.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, rustc_gdb_1031745::main () at src/main.rs:88
88 zzz(); // #break
$1 = &rustc_gdb_1031745::Foo<[u8]> 0x7fffffffdd88
$2 = &rustc_gdb_1031745::Foo<rustc_gdb_1031745::Foo<[u8]>> 0x7fffffffdd88
$3 = &rustc_gdb_1031745::Foo<dyn core::fmt::Debug> {pointer: 0x555555593034, vtable: 0x5555555a3000warning: (Internal error: pc 0x5555555a3000 in read in CU, but not in symtab.)
warning: (Error: pc 0x5555555a3000 in address map, but not in symtab.)
}
$4 = alloc::boxed::Box<rustc_gdb_1031745::Foo<dyn core::fmt::Debug>, alloc::alloc::Global> {pointer: 0x5555555a7ba0, vtable: 0x5555555a3000warning: (Internal error: pc 0x5555555a3000 in read in CU, but not in symtab.)
warning: (Error: pc 0x5555555a3000 in address map, but not in symtab.)
}
$5 = &(i32, i32, [i32]) [(0, 1, 0), (2, 3, 0)]
$6 = &(i32, i32, dyn core::fmt::Debug) {pointer: 0x5555555a3020, vtable: 0x5555555a3030warning: (Internal error: pc 0x5555555a3030 in read in CU, but not in symtab.)
warning: (Error: pc 0x5555555a3030 in address map, but not in symtab.)
}
---->8----

----8<---- good (gdb 12.1-4 from bookworm)
Breakpoint 1 at 0x81dd: file src/main.rs, line 88.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, rustc_gdb_1031745::main () at src/main.rs:88
88 zzz(); // #break
$1 = &rustc_gdb_1031745::Foo<[u8]> {data_ptr: 0x7fffffffdd88, length: 4}
$2 = &rustc_gdb_1031745::Foo<rustc_gdb_1031745::Foo<[u8]>> {data_ptr: 0x7fffffffdd88, length: 4}
$3 = &rustc_gdb_1031745::Foo<dyn core::fmt::Debug> {pointer: 0x555555593034, vtable: 0x5555555a3000}
$4 = alloc::boxed::Box<rustc_gdb_1031745::Foo<dyn core::fmt::Debug>, alloc::alloc::Global> {pointer: 0x5555555a7ba0, vtable: 0x5555555a3000}
$5 = &(i32, i32, [i32]) {data_ptr: 0x555555593038, length: 2}
$6 = &(i32, i32, dyn core::fmt::Debug) {pointer: 0x5555555a3020, vtable: 0x5555555a3030}
---->8----

Since this no longer (fatally) affects the build of rustc, I opted for a
lower severity this time around. It looks like an issue with symbol
mapping/lookup again. Feel free to adjust if you consider this blocking.

0: https://salsa.debian.org/fg/rustc-gdb-1031745

I am attaching last git reproducer to the bug

[Hector Oron (zumbi)]

Created attachment 14721
test case with readme (v2) - one last issue after proposed patch

[Tromey-b (tromey-b)]

(In reply to HectorOron from comment #11)

> Since this no longer (fatally) affects the build of rustc, I opted for a
> lower severity this time around. It looks like an issue with symbol
> mapping/lookup again. Feel free to adjust if you consider this blocking.

Would you mind submitting a separate bug for this?

[Tromey-b (tromey-b)]

Marking for 13.2

[Zixing Liu (liushuyu-011)]

Another refreshed build of Rust 1.67.1 has been uploaded to LL.
This build contains an updated patch that was iterated in the Rust upstream.
EVR for this build is 1.67.1+dfsg0ubuntu1-0ubuntu3.

You can find the source at https://git.launchpad.net/~canonical-foundations/ubuntu/+source/rustc/log/?h=merge-1.67 and binaries at this PPA: https://launchpad.net/~liushuyu-011/+archive/ubuntu/rust-updates-1.67.

[Tromey-b (tromey-b)]

*** Bug 30224 has been marked as a duplicate of this bug. ***

[Cvs-commit (cvs-commit)]

The master branch has been updated by Tom Tromey <email address hidden>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=5f056fcb3dce947447063f5ab225042177a59722

commit 5f056fcb3dce947447063f5ab225042177a59722
Author: Tom Tromey <email address hidden>
Date: Fri Feb 24 10:40:16 2023 -0700

    Fix crash in inside_main_func

    gdb 13.1 crashes while running the rust compiler's debugger tests.
    The crash has a number of causes.

    First, the rust compiler still uses the C++-like _Z mangling, but with
    its own twist -- some hex digits added to the end of a symbol. So,
    while gdb finds the correct name of "main":

    (top-gdb) p name
    $13 = 0x292e0c0 "rustc_gdb_1031745::main"

    It isn't found in the minsyms, because C++ demangling yields:

    [99] t 0x90c0 _ZN17rustc_gdb_10317454main17h5b5be7fe16a97225E section .text rustc_gdb_1031745::main::h5b5be7fe16a97225 zko06yobckx336v

    This could perhaps be fixed. I also filed a new PR to suggest
    preferring the linkage name of the main program.

    Next, the rust compiler emits both a DW_TAG_subprogram and a
    DW_TAG_namespace for "main". This happens because the file is named
    "main.rs" -- i.e., the bug is specific to the source file name. The
    crash also seems to require the nested function inside of 'main', at
    least for me. The namespace always is generated, but perhaps this
    changes the ordering in the DWARF.

    When inside_main_func looks up the main symbol, it finds the namespace
    symbol rather than the function. (I filed a bug about fixing gdb's
    symbol tables -- long overdue.)

    Meanwhile, as I think it's important to fix this crash sooner rather
    than later, this patch changes inside_main_func to check that the
    symbol that is found is LOC_BLOCK. This perhaps should have been done
    in the first place, anyway.

    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30158

[Cvs-commit (cvs-commit)]

The gdb-13-branch branch has been updated by Tom Tromey <email address hidden>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=562403d4f5483e0787219fdb88da72c707c8d7f6

commit 562403d4f5483e0787219fdb88da72c707c8d7f6
Author: Tom Tromey <email address hidden>
Date: Fri Feb 24 10:40:16 2023 -0700

    Fix crash in inside_main_func

    gdb 13.1 crashes while running the rust compiler's debugger tests.
    The crash has a number of causes.

    First, the rust compiler still uses the C++-like _Z mangling, but with
    its own twist -- some hex digits added to the end of a symbol. So,
    while gdb finds the correct name of "main":

    (top-gdb) p name
    $13 = 0x292e0c0 "rustc_gdb_1031745::main"

    It isn't found in the minsyms, because C++ demangling yields:

    [99] t 0x90c0 _ZN17rustc_gdb_10317454main17h5b5be7fe16a97225E section .text rustc_gdb_1031745::main::h5b5be7fe16a97225 zko06yobckx336v

    This could perhaps be fixed. I also filed a new PR to suggest
    preferring the linkage name of the main program.

    Next, the rust compiler emits both a DW_TAG_subprogram and a
    DW_TAG_namespace for "main". This happens because the file is named
    "main.rs" -- i.e., the bug is specific to the source file name. The
    crash also seems to require the nested function inside of 'main', at
    least for me. The namespace always is generated, but perhaps this
    changes the ordering in the DWARF.

    When inside_main_func looks up the main symbol, it finds the namespace
    symbol rather than the function. (I filed a bug about fixing gdb's
    symbol tables -- long overdue.)

    Meanwhile, as I think it's important to fix this crash sooner rather
    than later, this patch changes inside_main_func to check that the
    symbol that is found is LOC_BLOCK. This perhaps should have been done
    in the first place, anyway.

    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30158

    (cherry picked from commit 5f056fcb3dce947447063f5ab225042177a59722)

[Tromey-b (tromey-b)]

Fixed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package rustc - 1.67.1+dfsg0ubuntu1-0ubuntu2

---------------
rustc (1.67.1+dfsg0ubuntu1-0ubuntu2) lunar; urgency=medium

  * Fix an issue where rustc can not find the standard library
    - d/p/ubuntu-fix-sysroot-detection.patch: Fix sysroot detection
      logic so that rustc will find the correct library paths to
      search in.

rustc (1.67.1+dfsg0ubuntu1-0ubuntu1) lunar; urgency=medium

  * New upstream release 1.67.1 (LP: #2007642)
    - d/p/d-bootstrap-use-local-css.patch: Remove unnecessary hunks
    - d/p/ubuntu-disable-sse2-tests-on-i386.patch: Disable SSE2 tests
      on i386 builds
    - d/copyright: Update copyright information for new dependencies
      and also remove copyright data for deleted dependencies
    - d/patches: Refresh patches
    - d/control: Update vendored deps field
    - d/control: Refresh vendored dependencies list
    - d/source/lintian-overrides: add new test files to the list

rustc (1.66.1+dfsg0ubuntu1-0ubuntu1) lunar; urgency=medium

  * New upstream release 1.66.1 (LP: #2005122)
    - d/control: Switch to LLVM 15 for better optimizations
    - d/rules: Add workarounds for the new bootstrap tests
    - d/patches: Refresh patches to adapt to the new upstream changes
    - d/p/ubuntu-Revert-Use-constant-eval-to-do-strict-validity-check.patch:
      Dropped, no longer needed
    - d/copyright: refresh vendored crate license info
    - d/libstd-rust-1.66.lintian-overrides: update the override files
  * d/rules: Enable LLVM profiler support (LP: #2002694)

 -- Zixing Liu <email address hidden> Wed, 22 Feb 2023 21:38:58 -0700

[Tromey-b (tromey-b)]

*** Bug 29298 has been marked as a duplicate of this bug. ***