sw-patch upload patch failed with "AttributeError: 'cStringIO.StringO' object has no attribute 'fileno'"

[ccyu]

Title
-----
sw-patch upload patch failed with "AttributeError: 'cStringIO.StringO' object has no attribute 'fileno'"

Brief Description
-----------------
In all-in-one environment, use sw-patch upload function, got failed.

Severity
--------
Critical: System/Feature is not usable after the defect

Steps to Reproduce
------------------
1. $MY_REPO/stx/stx-update/cgcs-patch/bin/patch_build --id EXAMPLE_0001 --reboot-required N
   then got EXAMPLE_0001.patch
2. sudo sw-patch upload EXAMPLE_0001.patch
3. got failed with AttributeError: 'cStringIO.StringO' object has no attribute 'fileno' in /var/log/patching.log

Expected Behavior
------------------
patch will be uploaded and then applied.

Actual Behavior
----------------
uploaded failed

Reproducibility
---------------
<Reproducible/Intermittent>
State if the issue is 100% reproducible or intermittent. If it is intermittent, state the frequency of occurrence

System Configuration
--------------------
One node system

Branch/Pull Time/Commit
-----------------------
master branch 201b94d on 7 Sep
https://github.com/openstack/stx-update/blob/master/cgcs-patch/cgcs-patch/cgcs_patch/api/controllers/root.py

Timestamp/Logs
--------------
[wrsroot@controller-0 tmp(keystone_admin)]$ tail -20 /var/log/patching.log
2018-11-13T02:21:03: sw-patch-controller-daemon[5902]: patch_controller.py(2451): INFO: launching
2018-11-13T02:21:03: sw-patch-agent[15135]: patch_agent.py(189): INFO: Handling detailed query
2018-11-13T02:21:29: sw-patch-controller-daemon[5902]: patch_controller.py(2106): ERROR: An uncaught exception has occurred:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/lib64/python2.7/site-packages/cgcs_patch/api/app.py", line 43, in __call__
    return self.v1(environ, start_response)
  File "/usr/lib/python2.7/site-packages/pecan/middleware/recursive.py", line 56, in __call__
    return self.application(environ, start_response)
  File "/usr/lib/python2.7/site-packages/pecan/core.py", line 835, in __call__
    return super(Pecan, self).__call__(environ, start_response)
  File "/usr/lib/python2.7/site-packages/pecan/core.py", line 678, in __call__
    self.invoke_controller(controller, args, kwargs, state)
  File "/usr/lib/python2.7/site-packages/pecan/core.py", line 569, in invoke_controller
    result = controller(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/cgcs_patch/api/controllers/root.py", line 109, in upload
    src = fileitem.file.fileno()
AttributeError: 'cStringIO.StringO' object has no attribute 'fileno'
2018-11-13T02:21:32: sw-patch-controller-daemon[5902]: patch_controller.py(2273): INFO: Detected thread death. Terminating

/usr/lib64/python2.7/site-packages/cgcs_patch/api/controllers/root.py
 88 @expose('json')
 89 @expose('query.xml', content_type='application/xml')
 90 def upload(self):
 91 assert isinstance(request.POST['file'], cgi.FieldStorage)
 92 fileitem = request.POST['file']
 93
 94 if not fileitem.filename:
 95 return dict(error="Error: No file uploaded")
 96
 97 fn = '/scratch/' + os.path.basename(fileitem.filename)
 98 # This technique cannot copy a very large file. It
 99 # requires a lot of memory as all data from the
100 # source file is read into memory then written to
101 # the destination file one chunk
102 # open(fn, 'wb').write(fileitem.file.read())
103
104 # Copying file by chunks using OS system calls
105 # requires much less memory. A larger chunk
106 # size can be used to improve the copy speed;
107 # currently 64K chunk size is selected
108 dst = os.open(fn, os.O_WRONLY | os.O_CREAT)
109 src = fileitem.file.fileno()
110 size = 64 * 1024
111 n = size
112 while n >= size:
113 s = os.read(src, size)
114 n = os.write(dst, s)
115 os.close(dst)

[wrsroot@controller-0 tmp(keystone_admin)]$ systemctl status sw-patch-controller-daemon
● sw-patch-controller-daemon.service - TIS Patching Controller Daemon
   Loaded: loaded (/usr/lib/systemd/system/sw-patch-controller-daemon.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2018-11-13 02:21:02 UTC; 17s ago
  Process: 5803 ExecStop=/etc/init.d/sw-patch-controller-daemon stop (code=exited, status=0/SUCCESS)
  Process: 5895 ExecStart=/etc/init.d/sw-patch-controller-daemon start (code=exited, status=0/SUCCESS)
 Main PID: 5902 (python)
    Tasks: 4
   Memory: 54.3M
   CGroup: /system.slice/sw-patch-controller-daemon.service
           └─5902 python /usr/sbin/sw-patch-controller-daemon

[wrsroot@controller-0 tmp(keystone_admin)]$ sudo sw-patch upload EXAMPLE_0001.patch
An internal error has occurred. Please check /var/log/patching.log for details

[Don Penney (dpenney)]

The stx-update/extras/scripts/patch_build.sh utility is the designer-level tool for building patches, which calls the cgcs-patch/bin/patch_build util after setting required environment variables.

$ extras/scripts/patch_build.sh --help
Usage: patch_build [ <args> ] ... <rpm list>
Options:
        --id <id> Patch ID
        --release <version> Platform release version
        --status <status> Patch Status Code (ie. O, R, V)
        --unremovable Marks patch as unremovable
        --reboot-required <Y|N> Marks patch as reboot-required (default=Y)
        --summary <summary> Patch Summary
        --desc <description> Patch Description
        --warn <warnings> Patch Warnings
        --inst <instructions> Patch Install Instructions
        --req <patch_id> Required Patch
        --controller <rpm> New package for controller
        --compute <rpm> New package for compute node
        --compute-lowlatency <rpm> New package for compute-lowlatency node
        --storage <rpm> New package for storage node
        --controller-compute <rpm> New package for combined node
        --controller-compute-lowlatency <rpm> New package for lowlatency combined node
        --all-nodes <rpm> New package for all node types

You must specify the RPM file(s) to be included in the patch. The build command used as described in this bug report includes no RPM files, resulting in a malformed patch. And it seems this particular malformation is not handled well by the patching system.

[ccyu (indicoliteplus)]

I run the cases in subdir of stx-update/patch-scripts and met the errors.
⇒ ls
EXAMPLE_0001 EXAMPLE_AODH EXAMPLE_NEUTRON EXAMPLE_SERVICE test-patches
EXAMPLE_0002 EXAMPLE_HEAT EXAMPLE_NOVA EXAMPLE_SYSINV test-patches-suite-b
EXAMPLE_0003 EXAMPLE_MTCE EXAMPLE_RR EXAMPLE_VIM

Could you please show me a casual/normal case?
Or anything else prerequisite for making the patch?
Seems my system environment state is just okay.

Thanks.

[Ghada Khalil (gkhalil)]

Setting the priority to Low. As per the comments above, a malformed patch was used during the test.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/643438

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/643438
Committed: https://git.openstack.org/cgit/openstack/stx-update/commit/?id=8055fcbe517895e1aebc11633eae49e0d4d8ea9f
Submitter: Zuul
Branch: master

commit 8055fcbe517895e1aebc11633eae49e0d4d8ea9f
Author: Don Penney <email address hidden>
Date: Thu Mar 14 13:57:16 2019 -0400

    Protect against generating empty patch

    If a user attempts to generate a patch without specifying
    any packages, raise an exception to avoid malformed patches.

    Additionally, very small files (like a patch with no packages)
    appear to be handled differently by the API handler, passing
    a cStringIO object for small files and a file object for files
    that are about 1K or larger. For robustness, the upload
    handler is updated to do a hasattr('fileno') check in order
    to protect against an unhandled exception for small files.

    Change-Id: Ie1e730a183a27850ba8567f31aea603459b59d45
    Closes-Bug: 1803022
    Signed-off-by: Don Penney <email address hidden>

[Ghada Khalil (gkhalil)]

Adding stx.2019.05 tag as this fix was merged for that release