The decomp feature ROHC_DECOMP_FEATURE_CRC_REPAIR is not working

[Mohammad Abyan Abdullah]

Hi!!
    I've seen many crc errors due to network saturation at rohc packets. But none of them are going for repair. I've already called the rohc_decomp_set_features() to set the CRC_REPAIR. and no error shows after library initialization. But at decompressed error packets I don't see any of the packets are going for the CRC_REPAIR algo. Example for a packet here.

[general] [rohc_decomp.c:779 rohc_decompress3()] decompress the 46-byte packet #5012
[general] [rohc_decomp.c:3655 rohc_decomp_parse_padding()] skip 0 byte(s) of padding
[general] [rohc_decomp.c:1021 d_decode_header()] decompressor received 0 bytes of feedback for the same-side associated compressor
[general] [rohc_decomp.c:3618 rohc_decomp_decode_cid()] 1-byte large CID = 9
[general] [rohc_decomp.c:3772 rohc_decomp_find_context()] context with CID 9 found
[profile ID 1] [rohc_decomp.c:1177 d_decode_header()] 6 bits required for SN in feedback (6 bits required for RTT, 16 max)
[profile ID 1] [d_rtp.c:455 rtp_choose_uo1_variant()] UO-1* packet disambiguation: at least one IP header is IPv4 with context(RND) = 0, so parse as UO-1-ID or UO-1-TS
[profile ID 1] [d_rtp.c:469 rtp_choose_uo1_variant()] UO-1* packet disambiguation: T = 0, so parse as UO-1-ID
[profile ID 1] [rohc_decomp.c:1196 d_decode_header()] decode packet as 'RTP/UO-1-ID'
[profile ID 1] [rohc_decomp.c:1367 rohc_decomp_decode_pkt()] parse packet type 'RTP/UO-1-ID' (4)
[profile ID 1] [rohc_decomp_rfc3095.c:2109 parse_uo1id()] 5 IP-ID bits for IP header #1 = 0x1a
[profile ID 1] [rohc_decomp_rfc3095.c:2121 parse_uo1id()] 5 outer IP-ID bits = 0x1a
[profile ID 1] [rohc_decomp_rfc3095.c:2133 parse_uo1id()] 1-bit extension (X) = 1
[profile ID 1] [rohc_decomp_rfc3095.c:2137 parse_uo1id()] 4 SN bits = 0x6
[profile ID 1] [rohc_decomp_rfc3095.c:2142 parse_uo1id()] CRC-3 found in packet = 0x00
[profile ID 1] [rohc_decomp_rfc3095.c:2168 parse_uo1id()] first byte of extension = 0x2d
[profile ID 1] [rohc_decomp_rfc3095.c:4542 parse_extension0()] decode RTP/UO-1-ID extension 0
[profile ID 1] [rohc_decomp_rfc3095.c:4553 parse_extension0()] 3 bits of SN found in EXT-0 = 0x5
[profile ID 1] [rohc_decomp_rfc3095.c:4569 parse_extension0()] 3 bits of outer IP-ID found in EXT-0 = 0x5
[profile ID 1] [d_rtp.c:1469 rtp_parse_uo_remainder()] UDP checksum = 0xb308
[profile ID 1] [rohc_decomp.c:1385 rohc_decomp_decode_pkt()] ROHC payload (length = 40 bytes) starts at offset 6
[profile ID 1] [rohc_decomp_rfc3095.c:5589 rfc3095_decomp_decode_bits()] decoded SN = 1077 / 0x435 (nr bits = 7, bits = 53 / 0x35)
[profile ID 1] [rohc_decomp_rfc3095.c:5683 decode_ip_values_from_bits()] decoded outer TOS/TC = 0
[profile ID 1] [rohc_decomp_rfc3095.c:5696 decode_ip_values_from_bits()] decoded outer TTL/HL = 128
[profile ID 1] [rohc_decomp_rfc3095.c:5710 decode_ip_values_from_bits()] decoded outer protocol/NH = 17
[profile ID 1] [rohc_decomp_rfc3095.c:5726 decode_ip_values_from_bits()] decoded outer NBO = 1
[profile ID 1] [rohc_decomp_rfc3095.c:5739 decode_ip_values_from_bits()] decoded outer RND = 0
[profile ID 1] [rohc_decomp_rfc3095.c:5752 decode_ip_values_from_bits()] decoded outer SID = 0
[profile ID 1] [rohc_decomp_rfc3095.c:5811 decode_ip_values_from_bits()] decoded outer IP-ID = 0x120a (rnd = 0, nbo = 1, sid = 0, nr bits = 8, bits = 0xd5)
[profile ID 1] [rohc_decomp_rfc3095.c:5824 decode_ip_values_from_bits()] decoded outer DF = 0
[profile ID 1] [rohc_decomp_rfc3095.c:5840 decode_ip_values_from_bits()] decoded outer src address = 0a000636 (10.0.6.54)
[profile ID 1] [rohc_decomp_rfc3095.c:5856 decode_ip_values_from_bits()] decoded outer dst address = adc0250c (173.192.37.12)
[profile ID 1] [d_rtp.c:1529 rtp_decode_values_from_bits()] decoded UDP source port = 0x17e2
[profile ID 1] [d_rtp.c:1544 rtp_decode_values_from_bits()] decoded UDP destination port = 0x2b46
[profile ID 1] [d_rtp.c:1579 rtp_decode_values_from_bits()] decoded UDP checksum = 0xb308 (checksum present = 1)
[profile ID 1] [d_rtp.c:1593 rtp_decode_values_from_bits()] decoded RTP version = 2
[profile ID 1] [d_rtp.c:1607 rtp_decode_values_from_bits()] decoded R-P flag = 0
[profile ID 1] [d_rtp.c:1621 rtp_decode_values_from_bits()] decoded R-X flag = 0
[profile ID 1] [d_rtp.c:1635 rtp_decode_values_from_bits()] decoded CC = 0
[profile ID 1] [d_rtp.c:1651 rtp_decode_values_from_bits()] decoded RTP M flag = 0
[profile ID 1] [d_rtp.c:1665 rtp_decode_values_from_bits()] decoded R-PT = 18
[profile ID 1] [d_rtp.c:1668 rtp_decode_values_from_bits()] 0-bit TS delta = 0x0
[profile ID 1] [d_rtp.c:1705 rtp_decode_values_from_bits()] TS is deducted from SN
[general] [decomp_scaled_rtp_ts.c:444 ts_deduce_from_sn()] new TS_SCALED = 1077 (ref TS_SCALED = 954, new SN = 1077, ref SN = 954)
[general] [decomp_scaled_rtp_ts.c:450 ts_deduce_from_sn()] new TS = 344640 (TS_SCALED = 1077, TS_STRIDE = 320, TS_OFFSET = 0)
[profile ID 1] [d_rtp.c:1728 rtp_decode_values_from_bits()] decoded timestamp = 344640 / 0x54240 (nr bits = 0, bits = 0 / 0x0)
[profile ID 1] [d_rtp.c:1742 rtp_decode_values_from_bits()] decoded SSRC = 56234139
[profile ID 1] [rohc_decomp_rfc3095.c:5002 rfc3095_decomp_build_hdrs()] length of transport header = 20 bytes
[profile ID 1] [rohc_decomp_rfc3095.c:5164 build_uncomp_ipv4()] Total Length = 0x0050 (IHL * 4 + 60)
[profile ID 1] [rohc_decomp_rfc3095.c:5168 build_uncomp_ipv4()] IP checksum = 0x4591
[profile ID 1] [d_rtp.c:1785 rtp_build_uncomp_rtp()] UDP + RTP length = 0x003c
[profile ID 1] [rohc_decomp_rfc3095.c:5322 check_uncomp_crc()] CRC-3 on uncompressed header = 0x1
[profile ID 1] [rohc_decomp_rfc3095.c:5328 check_uncomp_crc()] CRC failure (computed = 0x01, packet = 0x00)
[profile ID 1] [rohc_decomp_rfc3095.c:5056 rfc3095_decomp_build_hdrs()] CRC detected a decompression failure for packet of type RTP/UO-1-ID in state Full Context and mode O-mode
[profile ID 1] [rohc_decomp.c:1506 rohc_decomp_decode_pkt()] CID 9: failed to build uncompressed headers (CRC failure)
[profile ID 1] [rohc_decomp.c:1519 rohc_decomp_decode_pkt()] CID 9: failed to build uncompressed headers (CRC failure)
[profile ID 1] [rohc_decomp.c:1248 d_decode_header()] failed to decompress packet (code = 4)
[general] [rohc_decomp.c:831 rohc_decompress3()] stay in O-mode as requested by user
[general] [rohc_decomp.c:888 rohc_decompress3()] packet decompression failed: CRC failure (4)
[general] [rohc_decomp.c:2030 rohc_decomp_feedback_nack()] CID 9: O-mode: Full Context state: RTP/UO-1-ID packet failed the CRC check
[general] [rohc_decomp.c:2149 rohc_decomp_feedback_nack()] avoid sending feedback too quickly (200 of 3200 with threshold 960)
[general] [rohc_decomp.c:2203 rohc_decomp_feedback_nack()] do not send a negative ACK
[general] [rohc_decomp.c:2295 rohc_decomp_feedback_nack()] do not perform a downward state transition

Regards
Abyan

[Didier Barvaux (didier-barvaux)]

Hello,

What version of the ROHC library do you use?

Regards,
Didier

[Mohammad Abyan Abdullah (c-admin-v)]

Hi!!
  I'm using version 2.0.0.

Regards
Abyan

[Didier Barvaux (didier-barvaux)]

Version 2.0.0 is not final yet, please tell me what Git version you use. Type 'git describe' in the rohc directory to get the information.

Didier

[Mohammad Abyan Abdullah (c-admin-v)]

Hi!!
  here is what git describes says.

fatal: No annotated tags can describe '0188c0cf7bba7a0b4b16b0c5a9de032e76768d45'.

Regards
Abyan

[Didier Barvaux (didier-barvaux)]

Hello,

I think I found something. If CRC repair is used once and fails, then it stays enabled and no further CRC repair may occur. Please try the following patch that resets the CRC repair state upon the next successful decompression of IR packet. Tell me if improves the situation for you.

Regards,
Didier

[Didier Barvaux (didier-barvaux)]

sorry typo: s/if improves/if it improves/

[Mohammad Abyan Abdullah (c-admin-v)]

Ok.
   I think it worked. But now I'm getting some wierd statistics like.

(bad CRC) packets= 12
corrected CRC failures = 566
corrected wrong SN updates = 566

Thanks for the patch.

Regards
Abyan

[Didier Barvaux (didier-barvaux)]

Hello,

Thanks for the feedback on the patch.

About the stats, is there some packet loss between the compressor and the décompressor ?

Regards,
Didier

[Mohammad Abyan Abdullah (c-admin-v)]

Yes
  there are few packet loss occurs on that path. It's about 1-5%.

Regards
Abyan

[Mohammad Abyan Abdullah (c-admin-v)]

I'm using rohc_decomp_get_last_packet_info() to get the information
and
last_packet_info.version_major = 0;
last_packet_info.version_minor = 1;
And
decomp_nr_corrected_crc_failures += last_packet_info.corrected_crc_failures;
decomp_nr_corrected_sn_wraparounds += last_packet_info.corrected_sn_wraparounds;
decomp_nr_corrected_wrong_sn_updates += last_packet_info.corrected_wrong_sn_updates;

After each packet decompress I'm updating the info. Don't think there might be any probs ...

Regards
Abyan.

[Didier Barvaux (didier-barvaux)]

Is the loss bursty? The ROHC protocol should handle random loss without CRC errors, but may lose synchronisation between compressor and decompressor if loss bursts are larger than the W-LSB window. If the loss is bursty, what is the length of the largest loss burst?

[Mohammad Abyan Abdullah (c-admin-v)]

Hi!!
The loss is not bursty but due to using packing along with it might be affected. I'm testing the environment with 5-10 packing frame at once.

Regards
Abyan

[Didier Barvaux (didier-barvaux)]

OK, try using the rohc_comp_set_wlsb_window_width() fonction to set the W-LSB window width to 11.

https://rohc-lib.org/support/documentation/API/rohc-doc-1.7.0/group__rohc__comp.html#ga2b68070dddbc038cd55490952a7b3fa4

[Mohammad Abyan Abdullah (c-admin-v)]

Hi!!
 I already use W-LSB window to 16.

Regards
Abyan

[Didier Barvaux (didier-barvaux)]

Hmm, try with max_lost_pkts_nr*packing+1

[Didier Barvaux (didier-barvaux)]

Or the nearest greater power of two.

[Mohammad Abyan Abdullah (c-admin-v)]

Ok.
   Changed W-LSB window to 32 and now here is the stats below.

decompression OK packets= 153585
decompression (no context) packets = 194
decompression (packet failed) packets = 94
decompression (bad CRC) packets = 46
decompression lost packets = 233
decompression misordered packets = 0
decompression duplicated packets = 0
decompression corrected CRC failures = 98333
decompression corrected SN wraparounds = 0
decompression corrected wrong SN updates = 98333

Packet loss was no more than 0-1%.

Regards
Abyan

[Didier Barvaux (didier-barvaux)]

Hello,

The stats are wrong. There are too many CRC corrections. The CRC repair stats retrieved with rohc_decomp_get_last_packet_info() are cumulative, so you should not add them to the values retrieved from previous calls to this function.

The fix is not straight forward. You cannot just use the last retrieved value, since the value is cumulative by context (not for all contexts) and the function gives no information about the context itself (the CID for example).

So, I propose you to introduce a new function rohc_decomp_get_context_info() that retrieves some information for a given context. The CRC repair stats are part of them. They are cumulative counters. So, you may retrieves the stats for all contexts at the very end of the program.

* The description of the function may be found there: https://github.com/didier-barvaux/rohc/commit/3bd7f2fa981da4c28a6e7bdd2069eba4dce4876e#diff-a7250f80721fe35bfcf368fbcc82ca93R2505
* The stats retrieved are found here: https://github.com/didier-barvaux/rohc/commit/3bd7f2fa981da4c28a6e7bdd2069eba4dce4876e#diff-c2a52ba517eeaac7846e6c06f4c04090R157
* an example of use may be found here: https://github.com/didier-barvaux/rohc/commit/3bd7f2fa981da4c28a6e7bdd2069eba4dce4876e#diff-cd3c1a82b2b16f40aa35b1d594c6b322R529

The patch is available in the new dev_fix_bug_1587011 branch, just after the previous patch that fixes the problem you reported previously: https://github.com/didier-barvaux/rohc/commits/dev_fix_bug_1587011
You can either retrieve that branch or manually download the patch from Github.

I also enhanced the rohc_decomp_get_general_info() function to retrieve the sames 3 counters but cumulated for all the contexts of one decompressor. Please have a look at https://github.com/didier-barvaux/rohc/commit/28ab10540374ffb118bf9bcfc050d076e20b5fc4

Please tell me if those functions fit your needs and what are the new computed stats in your use case.

Regards,
Didier

[Mohammad Abyan Abdullah (c-admin-v)]

Ok.
  I think the last one is suitable for my usage scenario. After updated that pacth here is the new stats below:

                        decompression OK packets,6816758
                        decompression (all) packets,1833
                        decompression (no context) packets,1566
                        decompression (packet failed) packets,0
                        decompressio (bad CRC) packets,267
                        decompression (error) packets,0
                        decompression (others) packets,0
                        decompression lost packets,1316
                        decompression misordered packets,288
                        decompression duplicated packets,0
                        decompression corrected CRC failures,89
                        decompression corrected SN wraparounds,0
                        decompression corrected wrong SN updates,89

Regards
Abyan

[Mohammad Abyan Abdullah (c-admin-v)]

Hi!
  Didier from rohc_decomp_get_last_packet_info (const struct rohc_decomp *const decomp,rohc_decomp_last_packet_info_t *const info) at decompressor end there is no packets with U-Mode and IR-State or FO-state found. Is it normal ? Here is the stats below

                        U-mode packets=0
                        O-mode packets=1455326
                        R-mode packets=0

                        IR state packets=0
                        FO state packets=0
                        SO state packets=1455326

Regards
Abyan

[Didier Barvaux (didier-barvaux)]

Hello,

No U- or R-mode is expected if you required that compressor/decompressor work in O-mode. No IR nor FO states is strange however. Did you catch the beginning of the stream?

Regards,
Didier