Compression errors with zlib 1.2.11

[maksis]

After updating zlib to version 1.2.10, transfers randomly fail because of compression errors. Such errors happening on uploader's side won't reported anywhere, the upload just gets disconnected.

Based on some debugging with AirDC++, the actual location where the error happens in zlib is https://github.com/madler/zlib/blob/master/deflate.c#L595

I'm not sure whether that's a bug in zlib or something intended as there are also breaking changes in version 1.2.10: https://github.com/madler/zlib/issues/190 (avail_out is not originally zero when the call is made)

Setting compression level at https://sourceforge.net/p/dcplusplus/code/ci/default/tree/dcpp/ZUtils.cpp#l59 to a non-zero value seems to help with avoiding the issue.

[eMTee (realprogger)]

I can confirm similar problems sporadically appearing during transfers between DC++ 0.864 and clients equipped with both zlib 1.2.10 and older versions.
Tested with downloading filelists, always the same transfers with same clients fail while the rest succeed so it probably depends on the data being compressed.

[maksis (maksis)]

I haven't experienced any errors with zlib 1.2.11 yet

[eMTee (realprogger)]

Nor have I so far.

[eMTee (realprogger)]

Until multiple people tried and failed to get my current filelist today. Which is not working so long as I have the compression enabled (with zlib 1.2.11).

[maksis (maksis)]

Yeah, it still happens with files as well. The next stable version AirDC++ will be released with zlib 1.2.8 regardless of what happens with this issue.

[poy (poy)]

patch that might work, cooked up by following the deflateParams doc. please test.

[maksis (maksis)]

No errors with a previously failing folder transfer after applying that patch

[eMTee (realprogger)]

No errors getting the same filelist from me which still fails without this patch.

[maksis (maksis)]

Should the check for "zs.avail_out == 0" still be kept in case zlib's error policy is changed again in future and for compatibility with older zlib versions? The code can't really be tied to a specific zlib version on Linux.

[Crise / MW (markuwil)]

Shouldn't a split like that be done using the preprocessor... unless you are absolutely certain that retaining that condition with the new version has no adverse effects.

zlib has the necessary constants to do this and the patch is small enough to make it a viable approach.

[maksis (maksis)]

That's one way to do it.

For anyone using the code on Linux, it's very much impossible to notice that such split is needed (before they actually run into problems and try to figure out what's wrong). I'd strongly avoid such caveats that rely on internal functionality changes of a library instead of actual compiler errors.

[poy (poy)]

this is actually easy to reproduce once one understands the causes. :)

pushed a test case that simulates compression being dynamically disabled, along with the fix I attached here (but with proper guards based on the zlib version).

[poy (poy)]

Fixed in DC++ 0.865.