Ironic

Ironic fails to deploy instance with pxe_ipmitool driver on system with Japanese locale

Bug #1488289 reported by Yuiko Takada on 2015-08-25

This bug affects 2 people

Affects		Status	Importance	Assigned to	Milestone
	Ironic	Fix Released	High	Ruby Loo	Ironic 4.2.1 "liberty"
	ironic-lib	Fix Released	High	Ruby Loo	ironic-lib 0.2.0 "0.2.0"

Bug Description

When I've tried to baremetal provisioning with pxe_ipmitool driver on RHELOSP7
with Japanese, ramdisk and kernel are successufully loaded and the server got a IP from neutron as well, but the deployment fails with iscsiadm commands with this error :

"InstanceDeployFailure: Deploy failed for instance 6d68e947-5ef2-4a4b-858f-7fae7b924d66. Error: 'ascii' codec can't decode byte 0xc2 in position 24: ordinal not in range(128)"

Here are the logs of ironic-conductor :

ironic.common.utils [-] Command stderr is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:85
oslo_concurrency.processutils [-] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/bd764188-1477-4c3a-81e4-a9e933fcec7a/disk execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:199
oslo_concurrency.processutils [-] CMD "env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/bd764188-1477-4c3a-81e4-a9e933fcec7a/disk" returned: 0 in 0.014s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:225
ironic.common.utils [-] Execution completed, command line is "env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/bd764188-1477-4c3a-81e4-a9e933fcec7a/disk" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83
ironic.common.utils [-] Command stdout is: "image: /var/lib/ironic/images/bd764188-1477-4c3a-81e4-a9e933fcec7a/disk
file format: raw
virtual size: 1.5G (1658585088 bytes)
disk size: 890M

" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:84
ironic.common.utils [-] Command stderr is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:85
ironic.drivers.modules.deploy_utils [-] Start destroy disk metadata for node bd764188-1477-4c3a-81e4-a9e933fcec7a. destroy_disk_metadata /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py:442
oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf dd if=/dev/zero of=/dev/disk/by-path/ip-192.168.10.56:3260-iscsi-iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a-lun-1 bs=512 count=36 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:199
oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf dd if=/dev/zero of=/dev/disk/by-path/ip-192.168.10.56:3260-iscsi-iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a-lun-1 bs=512 count=36" returned: 0 in 0.101s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:225
ironic.common.utils [-] Execution completed, command line is "dd if=/dev/zero of=/dev/disk/by-path/ip-192.168.10.56:3260-iscsi-iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a-lun-1 bs=512 count=36" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83
ironic.common.utils [-] Command stdout is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:84
oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 192.168.10.56:3260 -T iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a --logout execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:199
oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 192.168.10.56:3260 -T iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a --logout" returned: 0 in 0.104s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:225
ironic.common.utils [-] Execution completed, command line is "iscsiadm -m node -p 192.168.10.56:3260 -T iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a --logout" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83
ironic.common.utils [-] Command stdout is: "Logging out of session [sid: 3, target: iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a, portal: 192.168.10.56,3260]
Logout of [sid: 3, target: iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a, portal: 192.168.10.56,3260] successful.
" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:84

ironic.common.utils [-] Command stderr is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:85
oslo_concurrency.processutils [-] Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 192.168.10.56:3260 -T iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a -o delete execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:199
oslo_concurrency.processutils [-] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 192.168.10.56:3260 -T iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a -o delete" returned: 0 in 0.073s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:225
ironic.common.utils [-] Execution completed, command line is "iscsiadm -m node -p 192.168.10.56:3260 -T iqn-bd764188-1477-4c3a-81e4-a9e933fcec7a -o delete" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83
ironic.common.utils [-] Command stdout is: "" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:84

ironic.drivers.modules.iscsi_deploy [-] Deploy failed for instance 6d68e947-5ef2-4a4b-858f-7fae7b924d66. Error: 'ascii' codec can't decode byte 0xc2 in position 24: ordinal not in range(128)
ironic.common.states [-] Exiting old state 'deploying' in response to event 'fail' on_exit /usr/lib/python2.7/site-packages/ironic/common/states.py:177
ironic.common.states [-] Entering new state 'deploy failed' in response to event 'fail' on_enter /usr/lib/python2.7/site-packages/ironic/common/states.py:183
oslo_concurrency.processutils [-] Running cmd (subprocess): ipmitool -I lan -H 192.168.10.2 -L ADMINISTRATOR -U USERID -R 12 -N 5 -f /tmp/tmpyRYHjR power status execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:199
oslo_concurrency.processutils [-] CMD "ipmitool -I lan -H 192.168.10.2 -L ADMINISTRATOR -U USERID -R 12 -N 5 -f /tmp/tmpyRYHjR power status" returned: 0 in 0.696s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:225
ironic.common.utils [-] Execution completed, command line is "ipmitool -I lan -H 192.168.10.2 -L ADMINISTRATOR -U USERID -R 12 -N 5 -f /tmp/tmpyRYHjR power status" execute /usr/lib/python2.7/site-packages/ironic/common/utils.py:83

Tags:

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-08-25: Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/216478

Changed in ironic:
assignee:	nobody → Yuiko Takada (takada-yuiko)
status:	New → In Progress

Dmitry Tantsur (divius) on 2015-08-28

Changed in ironic:
importance:	Undecided → High

Revision history for this message

Jim Rollenhagen (jim-rollenhagen) wrote on 2015-09-03: Re: Ironic fails to deploy instance with pxe_ipmitool driver

Oops, was on the wrong bug :)

Changed in ironic:
status:	In Progress → Fix Committed
status:	Fix Committed → In Progress

Jim Rollenhagen (jim-rollenhagen) on 2015-09-15

Changed in ironic:
milestone:	none → 4.2.0

OpenStack Infra (hudson-openstack) on 2015-09-23

Changed in ironic:
assignee:	Yuiko Takada (takada-yuiko) → Ruby Loo (rloo)

Ruby Loo (rloo) on 2015-09-23

Changed in ironic:
assignee:	Ruby Loo (rloo) → Yuiko Takada (takada-yuiko)

Revision history for this message

Thiago Paiva Brito (outbrito) wrote on 2015-09-23:

I suspect this bug has something to do with what python is trying to output on logs.

aeva black (tenbrae) on 2015-09-23

summary:

- Ironic fails to deploy instance with pxe_ipmitool driver
+ Ironic fails to deploy instance with pxe_ipmitool driver on system with
+ Japanese locale

Jim Rollenhagen (jim-rollenhagen) on 2015-09-23

Changed in ironic:
milestone:	4.2.0 → none

Revision history for this message

Jim Rollenhagen (jim-rollenhagen) wrote on 2015-09-29:

Yuiko,

I'm having a really hard time reproducing this; could you please list steps to reproduce?

What I tried was following this guide and re-stacking, also making sure LANG is set correctly for the ironic-conductor process: http://www.server-world.info/en/note?os=Ubuntu_14.04&p=locale

Revision history for this message

Yuiko Takada (takada-yuiko) wrote on 2015-09-29:

Jim, thank you for trying, What I did is following steps:

1. (If there is no,) generate Japanese locale.
$ sudo locale-gen ja_JP.UTF-8
2. (If you don't have, ) get coreutils.mo and copy it to properly directory.
$ sudo cp coreutils.mo /usr/share/locale/ja/LC_MESSAGES/coreutils.mo
3. Set locale.
$ export LC_ALL=ja_JP.UTF-8

Thanks.

OpenStack Infra (hudson-openstack) on 2015-09-29

Changed in ironic:
assignee:	Yuiko Takada (takada-yuiko) → Ruby Loo (rloo)

Ruby Loo (rloo) on 2015-09-29

Changed in ironic:
assignee:	Ruby Loo (rloo) → Yuiko Takada (takada-yuiko)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-30: Fix proposed to ironic-lib (master)

Fix proposed to branch: master
Review: https://review.openstack.org/229434

Changed in ironic:
assignee:	Yuiko Takada (takada-yuiko) → Ruby Loo (rloo)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-30: Fix merged to ironic-lib (master)

Reviewed: https://review.openstack.org/229434
Committed: https://git.openstack.org/cgit/openstack/ironic-lib/commit/?id=618ab03b6071ff686dc8d385893c1461cf186527
Submitter: Jenkins
Branch: master

commit 618ab03b6071ff686dc8d385893c1461cf186527
Author: Ruby Loo <email address hidden>
Date: Wed Sep 30 13:47:20 2015 +0000

Use standard locale when executing 'dd' command

    When executing 'dd' command, baremetal provisioning fails on OS with
    Japanese because of ascii codec error:
     InstanceDeployFailure: Deploy failed for instance <uuid>.
     Error: 'ascii' codec can't decode byte 0xc2 in position 24:
            ordinal not in range(128)
    This fixes it by using standard locale when executing 'dd'.

Change-Id: I1248adb71489a818cfe1b5218f3bb6c961e66692
Closes-Bug:#1488289

Changed in ironic:
status:	In Progress → Fix Committed

Ruby Loo (rloo) on 2015-09-30

Changed in ironic-lib:
assignee:	nobody → Ruby Loo (rloo)
status:	New → Fix Committed
importance:	Undecided → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-01: Fix merged to ironic (master)

Reviewed: https://review.openstack.org/216478
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=8166c6e8ddd333ee9930d0d7d1afb7c2ee47fc4a
Submitter: Jenkins
Branch: master

commit 8166c6e8ddd333ee9930d0d7d1afb7c2ee47fc4a
Author: Yuiko Takada <email address hidden>
Date: Tue Aug 25 10:28:53 2015 +0900

Use standard locale when executing 'dd' command

    When executing 'dd' command, baremetal provisioning fails on OS with
    Japanese because of ascii codec error:
     InstanceDeployFailure: Deploy failed for instance <uuid>.
     Error: 'ascii' codec can't decode byte 0xc2 in position 24:
            ordinal not in range(128)
    This patch set fixes this bug by using standard locale when
    executing 'dd'.

    Change-Id: I8107975044cc9dec85e0d7571f99eaf5ea869891
    Closes-Bug:#1488289
    Co-Authored-By: Ruby Loo <email address hidden>

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-01: Fix proposed to ironic (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/229692

Doug Hellmann (doug-hellmann) on 2015-10-01

Changed in ironic-lib:
milestone:	none → 0.2.0
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-03: Fix merged to ironic (stable/liberty)

#10

Reviewed: https://review.openstack.org/229692
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=141c4819bd3059747ddb95166b990eb5ba576d52
Submitter: Jenkins
Branch: stable/liberty

commit 141c4819bd3059747ddb95166b990eb5ba576d52
Author: Yuiko Takada <email address hidden>
Date: Tue Aug 25 10:28:53 2015 +0900

Use standard locale when executing 'dd' command

    When executing 'dd' command, baremetal provisioning fails on OS with
    Japanese because of ascii codec error:
     InstanceDeployFailure: Deploy failed for instance <uuid>.
     Error: 'ascii' codec can't decode byte 0xc2 in position 24:
            ordinal not in range(128)
    This patch set fixes this bug by using standard locale when
    executing 'dd'.

    Change-Id: I8107975044cc9dec85e0d7571f99eaf5ea869891
    Closes-Bug:#1488289
    Co-Authored-By: Ruby Loo <email address hidden>
    (cherry picked from commit 8166c6e8ddd333ee9930d0d7d1afb7c2ee47fc4a)

tags:

added: in-stable-liberty

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-06: Related fix proposed to ironic (master)

#11

Related fix proposed to branch: master
Review: https://review.openstack.org/231367

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-06: Related fix proposed to ironic-lib (master)

#12

Related fix proposed to branch: master
Review: https://review.openstack.org/231369

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-06: Related fix merged to ironic-lib (master)

#13

Reviewed: https://review.openstack.org/231369
Committed: https://git.openstack.org/cgit/openstack/ironic-lib/commit/?id=9df46cb5500a68475fbde179ff6b642e2ab2e8c7
Submitter: Jenkins
Branch: master

commit 9df46cb5500a68475fbde179ff6b642e2ab2e8c7
Author: Naohiro Tamura <email address hidden>
Date: Tue Oct 6 17:23:31 2015 +0900

Use standard locale when executing 'parted' command

Not only 'dd' command but also 'parted' makes iscsi deploy fail in
Japanese environment. This patch applies the same fix as 'dd'.

     ironic.drivers.modules.iscsi_deploy [-] Deploy failed for instance
     <uuid>. Error: 'ascii' codec can't decode byte 0xe8 in position 20:
     ordinal not in range(128)

Related-Bug: #1488289
Change-Id: Ie3aedba60a4ed883ab24760947e65b8b3e9fa6b0

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-07: Related fix merged to ironic (master)

#14

Reviewed: https://review.openstack.org/231367
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=da5ddf4e703b6de9ef67420a0eb4baaa740980d7
Submitter: Jenkins
Branch: master

commit da5ddf4e703b6de9ef67420a0eb4baaa740980d7
Author: Naohiro Tamura <email address hidden>
Date: Tue Oct 6 16:36:47 2015 +0900

Use standard locale when executing 'parted' command

Not only 'dd' command but also 'parted' makes iscsi deploy fail in
Japanese environment. This patch applies the same fix as 'dd'.

     ironic.drivers.modules.iscsi_deploy [-] Deploy failed for instance
     <uuid>. Error: 'ascii' codec can't decode byte 0xe8 in position 20:
     ordinal not in range(128)

    Change-Id: If6c3fbd08ff380470d88b16605b44eb00c271352
    Related-Bug: #1488289
    Depends-On: Ie3aedba60a4ed883ab24760947e65b8b3e9fa6b0

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-13: Related fix proposed to ironic (stable/liberty)

#15

Related fix proposed to branch: stable/liberty
Review: https://review.openstack.org/234389

Jim Rollenhagen (jim-rollenhagen) on 2015-10-14

Changed in ironic:
milestone:	none → 4.2.1

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-14: Related fix merged to ironic (stable/liberty)

#16

Reviewed: https://review.openstack.org/234389
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=886bb91156556589d4d5a36dd969d0133831b1dc
Submitter: Jenkins
Branch: stable/liberty

commit 886bb91156556589d4d5a36dd969d0133831b1dc
Author: Naohiro Tamura <email address hidden>
Date: Tue Oct 6 16:36:47 2015 +0900

Use standard locale when executing 'parted' command

Not only 'dd' command but also 'parted' makes iscsi deploy fail in
Japanese environment. This patch applies the same fix as 'dd'.

     ironic.drivers.modules.iscsi_deploy [-] Deploy failed for instance
     <uuid>. Error: 'ascii' codec can't decode byte 0xe8 in position 20:
     ordinal not in range(128)

    Change-Id: If6c3fbd08ff380470d88b16605b44eb00c271352
    Related-Bug: #1488289
    (cherry picked from commit da5ddf4e703b6de9ef67420a0eb4baaa740980d7)

Doug Hellmann (doug-hellmann) on 2015-10-20

Changed in ironic:
status:	Fix Committed → Fix Released

Revision history for this message

Doug Hellmann (doug-hellmann) wrote on 2015-12-07: Fix included in openstack/ironic 4.3.0

#17

This issue was fixed in the openstack/ironic 4.3.0 release.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.