1. In `rust_demangle_callback`, "_R" at beginning causes `rdm.version == 0`[1][2], so `demangle_path` in the else-branch is called[3] with `rdm.sym == "YAbB2_"`, with `rdm.next` field initialized to 0.
2. Next char 'Y' causes `demangle_type` to be called[4] in a switch-case branch, with `rdm.next == 1`.
3. Next char 'A' causes `demangle_type` to be called first[5] to consume char 'b', then `demangle_const` is called[6] with `rdm.next == 3`.
4. Next char 'B' is consumed[7], and "2_" is parsed as `integer_62`[8] to be value 3, and this vlaue is assigned to `rdm.next`, then `demangle_const` is called recursively[9]. However, currently the situation is `rdm.next == 3` again, which is exactly same as last call to `demangle_const`, and this causes infinite recursion.
The simplified PoC:
$ echo _RYAbB2_ | ./binutils/cxxfilt
Segmentation fault
Root Cause Analysis:
1. In `rust_demangle_ callback` , "_R" at beginning causes `rdm.version == 0`[1][2], so `demangle_path` in the else-branch is called[3] with `rdm.sym == "YAbB2_"`, with `rdm.next` field initialized to 0.
2. Next char 'Y' causes `demangle_type` to be called[4] in a switch-case branch, with `rdm.next == 1`.
3. Next char 'A' causes `demangle_type` to be called first[5] to consume char 'b', then `demangle_const` is called[6] with `rdm.next == 3`.
4. Next char 'B' is consumed[7], and "2_" is parsed as `integer_62`[8] to be value 3, and this vlaue is assigned to `rdm.next`, then `demangle_const` is called recursively[9]. However, currently the situation is `rdm.next == 3` again, which is exactly same as last call to `demangle_const`, and this causes infinite recursion.
[1] https:/ /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L1356 /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L1362 /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L1453 /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L759 /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L934 /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L938 /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L1151 /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L1153 /github. com/bminor/ binutils- gdb/blob/ 626d0e40e55c35a 4f143b70def4987 34e8ed3c2a/ libiberty/ rust-demangle. c#L1158
[2] https:/
[3] https:/
[4] https:/
[5] https:/
[6] https:/
[7] https:/
[8] https:/
[9] https:/
Possible Fix:
The infinite recursion check added in this [commit](https:/ /github. com/bminor/ binutils- gdb/commit/ f10f8617a302f45 dae721eae0cd659 911f03d864) should also be applied to `demangle_const`