fail to use gzip when try to use Leverage compression accelerator (CentOS 7)

Bug #1862098 reported by Boxiang Zhu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Confirmed
Undecided
Unassigned

Bug Description

OS: CentOS Linux release 7.6.1810 (Core)
Kernel: 3.10.0-957.el7.x86_64
OpenStack Version: Train
Deploy Tool: kolla-ansible All-In-One Mode
You can see the globals.yml from attachment.

I just want to use the new feature http://specs.openstack.org/openstack/cinder-specs/specs/train/leverage-compression-accelerator.html
But after these steps, I met an error.
1. Download the cirros image and upload the image
2. Create a volume using this image as a bootable volume
3. Then at last, I use CLI "upload-to-image <volume-id> <new-image-name> --container-format compressed"

I got the error info from cinder-volume.log
===========================================

2020-02-06 10:29:11.012 37 INFO cinder.image.image_utils [req-9875a2fe-ed6e-4ad5-970e-3cd2167bb01e be2f14b985a244808f9a54e78d9f6fb7 2d3f871763e3434fb5d6597b9b644587 - default default] Converted 1024.00 MB image at 1024.00 MB/s
2020-02-06 10:29:12.384 37 ERROR cinder.image.accelerators.qat [req-9875a2fe-ed6e-4ad5-970e-3cd2167bb01e be2f14b985a244808f9a54e78d9f6fb7 2d3f871763e3434fb5d6597b9b644587 - default default] QATzip package is not installed.: ProcessExecutionError: Unexpected error while running command.
2020-02-06 10:29:14.185 37 ERROR cinder.volume.manager [req-9875a2fe-ed6e-4ad5-970e-3cd2167bb01e be2f14b985a244808f9a54e78d9f6fb7 2d3f871763e3434fb5d6597b9b644587 - default default] Upload volume to image encountered an error (image-id: 9e58527b-5fc5-4625-babf-fe2b829f2483).: CinderAcceleratorError: Cinder accelerator GZIP encountered an error while compressing/decompressing image.
2020-02-06 10:29:14.186 37 INFO cinder.message.api [req-9875a2fe-ed6e-4ad5-970e-3cd2167bb01e be2f14b985a244808f9a54e78d9f6fb7 2d3f871763e3434fb5d6597b9b644587 - default default] Creating message record for request_id = req-9875a2fe-ed6e-4ad5-970e-3cd2167bb01e
2020-02-06 10:29:14.304 37 WARNING cinder.volume.manager [req-9875a2fe-ed6e-4ad5-970e-3cd2167bb01e be2f14b985a244808f9a54e78d9f6fb7 2d3f871763e3434fb5d6597b9b644587 - default default] Deleting image in unexpected status: queued.: CinderAcceleratorError: Cinder accelerator GZIP encountered an error while compressing/decompressing image.
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server [req-9875a2fe-ed6e-4ad5-970e-3cd2167bb01e be2f14b985a244808f9a54e78d9f6fb7 2d3f871763e3434fb5d6597b9b644587 - default default] Exception during message handling: CinderAcceleratorError: Cinder accelerator GZIP encountered an error while compressing/decompressing image.
Command ['gzip', '-k', '/var/lib/cinder/conversion/tmp_K7_FI'] execution failed.
Volume compression failed while uploading to glance. GZIP compression command failed.
Reason: /bin/gzip: invalid option -- 'k'
Try `gzip --help' for more information.
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1687, in copy_volume_to_image
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server payload['message'] = six.text_type(error)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server self.force_reraise()
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1668, in copy_volume_to_image
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server image_service, image_meta)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 1595, in copy_volume_to_image
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server image_meta, tmp_file)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 716, in upload_volume
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server accel.compress_img(run_as_root=run_as_root)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/image/accelerator.py", line 91, in compress_img
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server run_as_root)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/image/accelerators/gzip.py", line 55, in compress_img
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server reason=ex.stderr)
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server CinderAcceleratorError: Cinder accelerator GZIP encountered an error while compressing/decompressing image.
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server Command ['gzip', '-k', '/var/lib/cinder/conversion/tmp_K7_FI'] execution failed.
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server Volume compression failed while uploading to glance. GZIP compression command failed.
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server Reason: /bin/gzip: invalid option -- 'k'
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server Try `gzip --help' for more information.
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server
2020-02-06 10:29:14.540 37 ERROR oslo_messaging.rpc.server
2020-02-06 10:29:16.795 37 INFO cinder.volume.manager [req-c9d45b21-4f50-46a2-8305-4f1b524a7a90 be2f14b985a244808f9a54e78d9f6fb7 2d3f871763e3434fb5d6597b9b644587 - default default] Deleted volume successfully.

Revision history for this message
Boxiang Zhu (bxzhu-5355) wrote :
Revision history for this message
Boxiang Zhu (bxzhu-5355) wrote :
Download full text (3.2 KiB)

I run `gzip --help` and `gzip --version`, result as followed:
[root@aio ~]# gzip --help
Usage: gzip [OPTION]... [FILE]...
Compress or uncompress FILEs (by default, compress FILES in-place).

Mandatory arguments to long options are mandatory for short options too.

  -c, --stdout write on standard output, keep original files unchanged
  -d, --decompress decompress
  -f, --force force overwrite of output file and compress links
  -h, --help give this help
  -l, --list list compressed file contents
  -L, --license display software license
  -n, --no-name do not save or restore the original name and time stamp
  -N, --name save or restore the original name and time stamp
  -q, --quiet suppress all warnings
  -r, --recursive operate recursively on directories
  -S, --suffix=SUF use suffix SUF on compressed files
  -t, --test test compressed file integrity
  -v, --verbose verbose mode
  -V, --version display version number
  -1, --fast compress faster
  -9, --best compress better
    --rsyncable Make rsync-friendly archive

With no FILE, or when FILE is -, read standard input.

Report bugs to <email address hidden>.
[root@aio ~]# gzip --version
gzip 1.5
Copyright (C) 2007, 2010, 2011 Free Software Foundation, Inc.
Copyright (C) 1993 Jean-loup Gailly.
This is free software. You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.

Written by Jean-loup Gailly.

I have a server Ubuntu 18.04, then I run the same commands, the result as followed:
$ gzip --help
Usage: gzip [OPTION]... [FILE]...
Compress or uncompress FILEs (by default, compress FILES in-place).

Mandatory arguments to long options are mandatory for short options too.

  -c, --stdout write on standard output, keep original files unchanged
  -d, --decompress decompress
  -f, --force force overwrite of output file and compress links
  -h, --help give this help
  -k, --keep keep (don't delete) input files
  -l, --list list compressed file contents
  -L, --license display software license
  -n, --no-name do not save or restore the original name and time stamp
  -N, --name save or restore the original name and time stamp
  -q, --quiet suppress all warnings
  -r, --recursive operate recursively on directories
  -S, --suffix=SUF use suffix SUF on compressed files
  -t, --test test compressed file integrity
  -v, --verbose verbose mode
  -V, --version display version number
  -1, --fast compress faster
  -9, --best compress better
  --rsyncable Make rsync-friendly archive

With no FILE, or when FILE is -, read standard input.

Report bugs to <email address hidden>.

# boxiang @ ZBX-DEV in ~ [11:04:48]
$ gzip --version
gzip 1.6
Copyright (C) 2007, 2010, 2011 Free Software Foundation, Inc.
Copyright (C) 1993 Jean-loup Gailly.
This is free software. You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted...

Read more...

Revision history for this message
Eric Harney (eharney) wrote :

The gzip command on el7 doesn't support "-k" (--keep).

Changed in cinder:
status: New → Confirmed
summary: fail to use gzip when try to use Leverage compression accelerator
+ (CentOS 7)
Revision history for this message
Eric Harney (eharney) wrote :

This option was added in gzip 1.6. CentOS 7 uses gzip 1.5.

Revision history for this message
Boxiang Zhu (bxzhu-5355) wrote :

Yes. So I think of two ways to solve the problem.
1. I do not use the "-k". Option "-k" is just to keep the orgin file. So we can copy it with "cp" command and then use "gzip" command without "-k" option;
2. We can just add some WARN info in the openstack docs, mark the version of gzip(>=1.6) to use;

anything else?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.