Comment 9 for bug 1986611

Thank you for sponsoring the patch into Kinetic Eric!

For bionic and focal, we decided to remove the python3-magic call for binary file detection, so for eg. this patch is added for bionic (and a similar one for focal) to revert to the older way. We still keep that function (file_is_binary() ) in utilities as is the case in 4.4, but the way it does binary file detection is reverted to 4.3 approach. Accordingly, we removed the magic stuff from requirements.txt and setup.py too. Please do let me know if you think it should remain in there for reasons we may have overlooked! We have let it remain though in snap/snapcraft.yaml and sos.spec.

~~~
Description: Revert to old way of binary file detection
 Reverting to older (4.3) way of detecting binary files
 because sos 4.4 uses python3-magic version 0.4.20 which is
 not present in bionic.
 .
 sosreport (4.4-1ubuntu0.18.04.1) bionic; urgency=medium
 .
   * New 4.4 upstream. (LP: #1986611)
 .
   * For more details, full release note is available here:
     - https://github.com/sosreport/sos/releases/tag/4.4
 .
   * Former patches, now fixed:
     - d/p/0002-fix-setup-py.patch
     - d/p/0003-mention-sos-help-in-sos-manpage.patch
 .
   * Remaining patches:
     - d/p/0001-debian-change-tmp-dir-location.patch
 .
   * New patches:
     - d/p/0002-revert-to-old-style-binary-file-detection.patch
Author: Nikhil Kshirsagar <email address hidden>
Bug-Ubuntu: https://bugs.launchpad.net/bugs/1986611

--- sosreport-4.4.orig/requirements.txt
+++ sosreport-4.4/requirements.txt
@@ -2,5 +2,4 @@ pycodestyle>=2.4.0
 coverage>=4.0.3
 Sphinx>=1.3.5
 pexpect>=4.0.0
-python_magic>=0.4.20
 pyyaml
--- sosreport-4.4.orig/setup.py
+++ sosreport-4.4/setup.py
@@ -107,7 +107,7 @@ setup(
     ],
     cmdclass=cmdclass,
     command_options=command_options,
- requires=['pexpect', 'python_magic', 'pyyaml']
+ requires=['pexpect', 'pyyaml']
     )

--- sosreport-4.4.orig/sos/utilities.py
+++ sosreport-4.4/sos/utilities.py
@@ -19,7 +19,6 @@ import tempfile
 import threading
 import time
 import io
-import magic

 from contextlib import closing
 from collections import deque
@@ -75,17 +74,14 @@ def file_is_binary(fname):
     :returns: True if binary, else False
     :rtype: ``bool``
     """
- try:
- _ftup = magic.detect_from_filename(fname)
- _mimes = ['text/', 'inode/']
- return (
- _ftup.encoding == 'binary' and not
- any(_ftup.mime_type.startswith(_mt) for _mt in _mimes)
- )
- except Exception:
- # if for some reason this check fails, don't blindly remove all files
- # but instead rely on other checks done by the component
- return False
+ with open(fname, 'tr') as tfile:
+ try:
+ # when opened as above (tr), reading binary content will raise
+ # an exception
+ tfile.read(1)
+ return False
+ except UnicodeDecodeError:
+ return True

 def find(file_pattern, top_dir, max_depth=None, path_pattern=None):
~~~