Juniper Openstack

SM:R3.0.3.x build 1: Upload contrail package may fail if there are old repo files there

Bug #1634003 reported by sundarkh
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.0
Won't Fix
Medium
Dheeraj Gautam
R3.0.3.x
Won't Fix
Medium
Dheeraj Gautam
R3.1
Fix Committed
Medium
Dheeraj Gautam
Juniper Openstack r3.1.1.0
R3.2
Fix Committed
Medium
Dheeraj Gautam
Juniper Openstack r3.2.0.0-fcs "r3.2.0.0"
Trunk
Fix Committed
Medium
Dheeraj Gautam
Juniper Openstack r4.0.0.0-fcs "r4.0.0.0"

Bug Description

SM:R3.0.3.x build 1: upload contrail package times out wth Error: (28, 'Operation timed out after 120000 milliseconds with 0 bytes received')

1) Install R3.0.3.x build Kilo SM
2) Upload ubuntu-14-04 image, uplaod happens succesfully
3) Upload the contrail package, it fails to get uploaded in to SM database

oot@nodej5:~# server-manager show all
{
    "cluster": [],
    "image": [
        {
            "id": "ubuntu-14-04"
        }
    ],
    "server": [],
    "tag": {}
}

COmamnd USed:

        server-manager-client upload-image --id ubuntu14kilo1 --version 1 --category package --type contrail-ubuntu-package --file_name /cs-shared/github-build/R3.0.3.x/1/ubuntu-14-04/kilo/contrail-install-packages_3.0.3.1-1~kilo_all.deb

Error :

Error: (28, 'Operation timed out after 120000 milliseconds with 0 bytes received')

Error Trace:

2016-10-16 23:02:00,383-DEBUG-server_mgr_main.py:1717-form_operartion_data(): function_start"
"2016-10-16 23:02:00,550-DEBUG-server_mgr_main.py:1743-upload_image(): upload_image"
"2016-10-16 23:04:26,153-ERROR-server_mgr_main.py:2160-_create_deb_repo(): create_deb_repo: error 254 when executing"reprepro includedeb contrail /var/www/html/contrail/repo/ubuntu14kilo1/*.deb""
"2016-10-16 23:04:26,177-ERROR-server_mgr_main.py:3162-log_trace(): *****TRACEBACK-START*****"
"2016-10-16 23:04:26,178-ERROR-server_mgr_main.py:3166-log_trace(): Traceback (most recent call last):
"
"2016-10-16 23:04:26,179-ERROR-server_mgr_main.py:3166-log_trace(): File "/opt/contrail/server_manager/server_mgr_main.py", line 1792, in upload_image
    image_id, image_type, image_version, dest)
"
"2016-10-16 23:04:26,180-ERROR-server_mgr_main.py:3166-log_trace(): File "/opt/contrail/server_manager/server_mgr_main.py", line 2255, in _create_repo
    raise(e)
"
"2016-10-16 23:04:26,181-ERROR-server_mgr_main.py:3166-log_trace(): ServerMgrException: u'create_deb_repo: error 254 when executing"reprepro includedeb contrail /var/www/html/contrail/repo/ubuntu14kilo1/*.deb"'
"
"2016-10-16 23:04:26,181-ERROR-server_mgr_main.py:3167-log_trace(): *****TRACEBACK-END******"
"

NOte :

1) SM Box has 24G space
root@nodej5:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 24G 4.0K 24G 1% /dev
tmpfs 4.8G 1.2M 4.8G 1% /run
/dev/dm-0 778G 397G 342G 54% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 24G 0 24G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sdb1 236M 39M 185M 18% /boot
10.204.217.151:/home/stack/cs-shared 3.7T 3.0T 564G 85% /cs-shared
10.204.216.49:/home/stack/cs-shared 852G 621G 188G 77% /cs-shared-test

sundarkh (sundar-kh)
tags: added: server-manager
Revision history for this message
sundarkh (sundar-kh) wrote :

Tried add image

it fails with following error

root@nodej5:~/sm_files# server-manager add image
id:ubuntu14kilo1
version (Specify version for this image) : 1
category (image/package) : package
type (ubuntu/centos/redhat/esxi5.1/esxi5.5/contrail-ubuntu-package/contrail-centos-package/contrail-storage-ubuntu-package) : contrail-ubuntu-package
path (complete path where image file is located on server) : /root/sm_files/contrail-install-packages_3.0.3.1-1~kilo_all.deb
kickstart (kickstart file for base image) :
kickseed (kickseed file for base image) :
More image(s) to input? (y/N)n
{
    "return_code": 8,
    "return_data": null,
    "return_msg": "create_deb_repo: error 254 when executing\"reprepro includedeb contrail /var/www/html/contrail/repo/ubuntu14kilo1/*.deb\""
}

Trace

"2016-10-17 02:30:03,367-DEBUG-server_mgr_main.py:1264-get_image(): get_image"
"2016-10-17 02:30:03,368-DEBUG-server_mgr_main.py:1047-validate_smgr_request(): validate_smgr_request"
"2016-10-17 02:30:03,369-DEBUG-server_mgr_main.py:699-validate_smgr_get(): match key returned: {'id': ['ubuntu14kilo1']}"
"2016-10-17 02:30:03,370-DEBUG-server_mgr_main.py:701-validate_smgr_get(): select keys: None"
"2016-10-17 02:30:03,370-DEBUG-server_mgr_main.py:1072-validate_smgr_request(): ret_val_data returned: {'status': 0, 'match_key': 'id', 'match_value': 'ubuntu14kilo1', 'select': None, 'detail': True}"
"2016-10-17 02:30:03,371-DEBUG-server_mgr_main.py:1296-get_image(): None"
"2016-10-17 02:30:30,279-DEBUG-server_mgr_main.py:1358-put_image(): add_image"
"2016-10-17 02:30:30,281-DEBUG-server_mgr_main.py:1047-validate_smgr_request(): validate_smgr_request"
"2016-10-17 02:30:30,281-INFO-server_mgr_main.py:230-merge_dict(): category already present in dict d1,Retaining value package against "
"2016-10-17 02:30:30,282-INFO-server_mgr_main.py:234-merge_dict(): adding kickstart:None"
"2016-10-17 02:30:30,283-INFO-server_mgr_main.py:234-merge_dict(): adding kickseed:None"
"2016-10-17 02:30:30,284-INFO-server_mgr_main.py:230-merge_dict(): version already present in dict d1,Retaining value 1 against "
"2016-10-17 02:30:30,285-INFO-server_mgr_main.py:230-merge_dict(): path already present in dict d1,Retaining value /root/sm_files/contrail-install-packages_3.0.3.1-1~kilo_all.deb against "
"2016-10-17 02:30:30,285-INFO-server_mgr_main.py:230-merge_dict(): type already present in dict d1,Retaining value contrail-ubuntu-package against "
"2016-10-17 02:30:30,286-INFO-server_mgr_main.py:230-merge_dict(): id already present in dict d1,Retaining value ubuntu14kilo1 against "
"2016-10-17 02:30:30,287-DEBUG-server_mgr_main.py:1072-validate_smgr_request(): ret_val_data returned: {'status': 1}"
"2016-10-17 02:31:28,669-ERROR-server_mgr_main.py:2160-_create_deb_repo(): create_deb_repo: error 254 when executing"reprepro includedeb contrail /var/www/html/contrail/repo/ubuntu14kilo1/*.deb""

Jeba Paulaiyan (jebap)
tags: added: blocker
Dheeraj Gautam (dgautam)
tags: removed: blocker
Revision history for this message
Dheeraj Gautam (dgautam) wrote :

Looked at the setup, with build 3.0.3-2, time out is not coming, images are being added or a failure is reported.

Failure while addition of image happened due to old directories into /var/www/html/contrail/repo. There was no image-id but /var/www/html/contrail/repo/image_id had some old files and causing reprepo to fail.

Dheeraj Gautam (dgautam)
summary: - SM:R3.0.3.x build 1: upload contrail package times out wth Error: (28,
- 'Operation timed out after 120000 milliseconds with 0 bytes received')
+ SM:R3.0.3.x build 1: Upload contrail package may fail if there are old
+ repo files there
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.1

Review in progress for https://review.opencontrail.org/25590
Submitter: Dheeraj Gautam (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/25590
Committed: http://github.org/Juniper/contrail-server-manager/commit/1ec50cb648313e44d8fa64f070514224963f6eff
Submitter: Zuul
Branch: R3.1

commit 1ec50cb648313e44d8fa64f070514224963f6eff
Author: Dheeraj Gautam <email address hidden>
Date: Tue Nov 1 22:34:18 2016 -0700

SM-Add-Image-Stage: Remove dirs first then add packages/images.

Closes-Bug: #1634003

remove directory before we do mkdir for directories.

PATCH2:
======
dist_util.remove_tree() requires directary to be exist. so moving to rm -fr.

Change-Id: I07066b96d48d5977107b0b8f117a62d0d0e6e7c9

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/26063
Submitter: Dheeraj Gautam (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/26064
Submitter: Dheeraj Gautam (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/26064
Committed: http://github.org/Juniper/contrail-server-manager/commit/e04aa1f7e258371ac4609cc625320f5cb787e968
Submitter: Zuul
Branch: master

commit e04aa1f7e258371ac4609cc625320f5cb787e968
Author: Dheeraj Gautam <email address hidden>
Date: Tue Nov 1 22:34:18 2016 -0700

SM-Add-Image-Stage: Remove dirs first then add packages/images.

Closes-Bug: #1634003

remove directory before we do mkdir for directories.

PATCH2:
======
dist_util.remove_tree() requires directary to be exist. so moving to rm -fr.

Change-Id: I07066b96d48d5977107b0b8f117a62d0d0e6e7c9

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/26063
Committed: http://github.org/Juniper/contrail-server-manager/commit/9826eaa6682ba83e584d52892b4e314bfe66bf82
Submitter: Zuul
Branch: R3.2

commit 9826eaa6682ba83e584d52892b4e314bfe66bf82
Author: Dheeraj Gautam <email address hidden>
Date: Tue Nov 1 22:34:18 2016 -0700

SM-Add-Image-Stage: Remove dirs first then add packages/images.

Closes-Bug: #1634003

remove directory before we do mkdir for directories.

PATCH2:
======
dist_util.remove_tree() requires directary to be exist. so moving to rm -fr.

Change-Id: I07066b96d48d5977107b0b8f117a62d0d0e6e7c9

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.