Tempest failing on os_nova role on CentOS 7

Bug #1692044 reported by Major Hayden
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack-Ansible
Fix Released
High
Unassigned

Bug Description

The tempest run in the CI jobs for the os_nova role is failing due to an issue in libvirtd and/or the kernel ( https://gist.github.com/major/8ade5d1187b0831f9ec1abf1cf479f03 ):

------------[ cut here ]------------
WARNING: at net/core/dev.c:2402 skb_warn_bad_offload+0xcd/0xda()
vif: caps=(0x0000000100054803, 0x00001001001f480b) len=1426 data_len=1374 gso_size=1343 gso_type=1 ip_summed=3
Modules linked in: nf_conntrack_netlink xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_connmark vxlan ip6_udp_tunnel udp_tunnel xt_comment nfsd auth_rpcgss nfs_acl lockd grace sunrpc veth btrfs zlib_deflate raid6_pq xor loop xt_CHECKSUM vhost_net tun vhost macvtap macvlan iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_vs libcrc32c ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_snapshot dm_bufio dm_multipath br_netfilter 8021q garp mrp ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw
 ebtable_filter ebtables ip6table_filter xenfs ip6_tables xen_privcmd iptable_filter dm_mirror dm_region_hash dm_log dm_mod intel_powerclamp intel_rapl iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd i2c_piix4 pcspkr parport_pc parport ip_tables ext4 mbcache jbd2 ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix xen_netfront xen_blkfront crct10dif_pclmul libata crct10dif_common crc32c_intel i2c_core serio_raw floppy fjes
CPU: 6 PID: 10840 Comm: sshd Tainted: G W ------------ 3.10.0-514.16.1.el7.x86_64 #1
Hardware name: Xen HVM domU, BIOS 4.1.5 11/28/2013
 ffff8801d8a6b7b0 000000007a202253 ffff8801d8a6b768 ffffffff81686ac3
 ffff8801d8a6b7a0 ffffffff81085cb0 ffff8801c1eb1cf8 ffff8801f3510000
 0000000000000001 0000000000000003 ffff8801c1eb1cf8 ffff8801d8a6b808
Call Trace:
 [<ffffffff81686ac3>] dump_stack+0x19/0x1b
 [<ffffffff81085cb0>] warn_slowpath_common+0x70/0xb0
 [<ffffffff81085d4c>] warn_slowpath_fmt+0x5c/0x80
 [<ffffffff8131ed33>] ? ___ratelimit+0x93/0x100
 [<ffffffff816894e8>] skb_warn_bad_offload+0xcd/0xda
 [<ffffffff8156e5cc>] skb_checksum_help+0x17c/0x190
 [<ffffffffa0470079>] checksum_tg+0x29/0x30 [xt_CHECKSUM]
 [<ffffffffa0058cb0>] ipt_do_table+0x2e0/0x710 [ip_tables]
 [<ffffffffa0058d07>] ? ipt_do_table+0x337/0x710 [ip_tables]
 [<ffffffffa031025b>] ? tcp_packet+0x3ab/0xa30 [nf_conntrack]
 [<ffffffffa0058d07>] ? ipt_do_table+0x337/0x710 [ip_tables]
 [<ffffffffa0302043>] iptable_mangle_hook+0x43/0x130 [iptable_mangle]
 [<ffffffff815a6fd0>] nf_iterate+0x70/0xb0
 [<ffffffff815a70b8>] nf_hook_slow+0xa8/0x110
 [<ffffffff815b72de>] ip_output+0xce/0xe0
 [<ffffffff815b5e20>] ? ip_fragment.constprop.54+0x90/0x90
 [<ffffffff815b4f41>] ip_local_out_sk+0x31/0x40
 [<ffffffff815b52b3>] ip_queue_xmit+0x143/0x3a0
 [<ffffffff815cea3f>] tcp_transmit_skb+0x4af/0x990
 [<ffffffff815cf07a>] tcp_write_xmit+0x15a/0xce0
 [<ffffffff815cfe7e>] __tcp_push_pending_frames+0x2e/0xc0
 [<ffffffff815be72c>] tcp_push+0xec/0x120
 [<ffffffff815c1fa8>] tcp_sendmsg+0xc8/0xc40
 [<ffffffff815ed254>] inet_sendmsg+0x64/0xb0
 [<ffffffff812abdd3>] ? selinux_socket_sendmsg+0x23/0x30
 [<ffffffff81554e07>] sock_aio_write+0x157/0x180
 [<ffffffff81212fd1>] ? set_fd_set+0x21/0x30
 [<ffffffff811fdf3d>] do_sync_write+0x8d/0xd0
 [<ffffffff811fe8a5>] vfs_write+0x1b5/0x1e0
 [<ffffffff811ff2cf>] SyS_write+0x7f/0xe0
 [<ffffffff81697189>] system_call_fastpath+0x16/0x1b
---[ end trace 1e801294230dc626 ]---
------------[ cut here ]------------

Revision history for this message
Major Hayden (rackerhacker) wrote :

Downgrading the kernel to the previous yields a core dump in libvirt. :/

Changed in openstack-ansible:
status: New → Confirmed
importance: Undecided → Medium
importance: Medium → High
Revision history for this message
Major Hayden (rackerhacker) wrote :

This was fixed by letting libvirt determine the cpu type.

Changed in openstack-ansible:
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.