puppetlabs-firefall breaks down on parsing Kubespray/Calico iptables rules

Bug #1733628 reported by Jiří Stránský
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Jiří Stránský

Bug Description

Deployment goes fine including deploying step 2 of external_deploy_tasks w/ Kubernetes, but when step 2 puppet kicks in, puppetlabs-firewall cannot parse the iptables rules produced by Calico, and fails the Puppet apply this way:

Error: Failed to apply catalog: Parser error: keys (4) and values (8) count mismatch on line: -A cali-PREROUTING -m comment --comment \"cali:5TQcm-i_T8rVGEEa\" -m comment --comment \"Host endpoint polic
y accepted packet.\" -m mark --mark 0x1000000/0x1000000 -j ACCEPT

I've also seen:

Error: Failed to apply catalog: Parser error: keys (6) and values (8) count mismatch on line: -A KUBE-SERVICES ! -s 10.233.64.0/18 -d 10.233.0.1/32 -p tcp -m comment --comment \"default/kubernetes:https
 cluster IP\" -m tcp --dport 443 -j KUBE-MARK-MASQ

More complete puppet output:

fatal: [192.168.24.6]: FAILED! => {
    "(outputs.stderr|default('')).split('\n')|union(outputs.stdout_lines|default([]))": [
        "exception: connect failed",
        "Warning: Facter: Fact resolution fact='systemd_internal_services', resolution='<anonymous>' resolved to an invalid value: Expected disabled to be one of [Integer, Float, TrueClass, FalseClass, NilClass,
 String, Array, Hash], but was Symbol",
        "Warning: Undefined variable 'deploy_config_name'; ",
        " (file & line not available)",
        "Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Bool. There is further documentation for validate_legacy function in the README. at [\"/etc/puppe
t/modules/ntp/manifests/init.pp\", 54]:[\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\", 29]",
        " (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:25:in `deprecation')",
        "Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Absolute_Path. There is further documentation for validate_legacy function in the README. at [\"/
etc/puppet/modules/ntp/manifests/init.pp\", 55]:[\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\", 29]",
        "Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at [\"/etc/pup
pet/modules/ntp/manifests/init.pp\", 56]:[\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\", 29]",
        "Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Array. There is further documentation for validate_legacy function in the README. at [\"/etc/pupp
et/modules/ntp/manifests/init.pp\", 66]:[\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\", 29]",
        "Warning: This method is deprecated, please use the stdlib validate_legacy function, with Pattern[]. There is further documentation for validate_legacy function in the README. at [\"/etc/puppet/modules/n
tp/manifests/init.pp\", 68]:[\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\", 29]",
        "Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Numeric. There is further documentation for validate_legacy function in the README. at [\"/etc/pu
ppet/modules/ntp/manifests/init.pp\", 76]:[\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\", 29]",
        "Warning: ModuleLoader: module 'timezone' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules",
        "Warning: ModuleLoader: module 'ssh' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules",
        "Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Hash. There is further documentation for validate_legacy function in the README. at [\"/etc/puppe
t/modules/ssh/manifests/server.pp\", 12]:[\"/var/lib/tripleo-config/puppet_step_config.pp\", 12]",
        "Error: Failed to apply catalog: Parser error: keys (6) and values (8) count mismatch on line: -A KUBE-SERVICES ! -s 10.233.64.0/18 -d 10.233.0.1/32 -p tcp -m comment --comment \"default/kubernetes:https
 cluster IP\" -m tcp --dport 443 -j KUBE-MARK-MASQ",
        "Notice: hiera(): Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend",
        "Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked.",
        "Notice: Compiled catalog for overcloud-compute-0.localdomain in environment production in 2.02 seconds",
        "Notice: /Stage[main]/Tripleo::Profile::Base::Kernel/Kmod::Load[nf_conntrack_proto_sctp]/Exec[modprobe nf_conntrack_proto_sctp]/returns: executed successfully",
        "Notice: /Stage[main]/Tripleo::Profile::Base::Kernel/Sysctl::Value[net.netfilter.nf_conntrack_max]/Sysctl_runtime[net.netfilter.nf_conntrack_max]/val: val changed '131072' to '500000'",
        "Notice: /Stage[main]/Tripleo::Profile::Base::Kernel/Sysctl::Value[net.nf_conntrack_max]/Sysctl_runtime[net.nf_conntrack_max]/val: val changed '131072' to '500000'"
    ],
    "failed_when_result": true
}

Tags: containers
Revision history for this message
Alex Schultz (alex-schultz) wrote :

If i recall, the puppet-firewall needs comments in a specific format so it doesn't stomp on them. Either no comments or it has to fit in the "\d+: <comment>" format. Or something to that effect.

Changed in tripleo:
milestone: none → queens-3
Revision history for this message
Alex Schultz (alex-schultz) wrote :
Revision history for this message
Jiří Stránský (jistr) wrote :

I debugged and found the cause, see

https://tickets.puppetlabs.com/browse/MODULES-6029

Revision history for this message
Jiří Stránský (jistr) wrote :

@Alex: Ah sorry i didn't refresh the page and didn't see what you posted. Will browse those links too.

Revision history for this message
Jiří Stránský (jistr) wrote :

@Alex: i'm not sure the unmanaged rule patch is directly related -- i may be wrong but the way i read it, having the rule considered unmanaged will not exclude it from being run through the parser.

Revision history for this message
Jiří Stránský (jistr) wrote :

I posted https://github.com/puppetlabs/puppetlabs-firewall/pull/738 and it's merged, it's in RDO trunk repos, we should have it available after a promote happens.

Changed in tripleo:
assignee: nobody → Jiří Stránský (jistr)
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.