Requested MTU is too big, maximum is 1500

Bug #1826729 reported by Noam Angel
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Alex Schultz

Bug Description

when installing undercloud with local_mtu > 1500 openstack udnercloud install failing, also if undercloud install with mtu 1500 we can not deploy overcloud with mtu larger then 1500 because we get same issue that neutron cannot create tenant network, global_physnet_mtu should be set to 0.

this seems to because of change:
https://review.opendev.org/#/c/612533/

Workaround:
echo " TenantNetPhysnetMtu: 0" | sudo tee -a /usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml

Error:
TASK [Output for sync deployment UndercloudCtlplaneNetworkDeployment] **********
fatal: [cloudx-uc3]: FAILED! => {
    "msg": [
        {
            "stderr": [
                "[2019-04-28 09:15:26,050] (heat-config) [DEBUG] Running /usr/libexec/heat-config/hooks/script < /var/lib/heat-config/deployed/f6d13638-0cff-48d1-ab4b-d89e714ef153.json",
                "[2019-04-28 09:15:30,370] (heat-config) [INFO] |-",
                " {\"deploy_stdout\": \"ERROR: Network create/update failed.\\n\", \"deploy_stderr\": \"Traceback (most recent call last):\\n File \\\"/var/lib/heat-config/heat-config-script/f6d13638-0cff-48d1-ab4b-d89e714ef153\\\", line 283, in <module>\\n network = _ensure_neutron_network(sdk)\\n File \\\"/var/lib/heat-config/heat-config-script/f6d13638-0cff-48d1-ab4b-d89e714ef153\\\", line 50, in _ensure_neutron_network\\n mtu=CONF['mtu'])\\n File \\\"/usr/lib/python2.7/site-packages/openstack/network/v2/_proxy.py\\\", line 1204, in create_network\\n return self._create(_network.Network, **attrs)\\n File \\\"/usr/lib/python2.7/site-packages/openstack/proxy.py\\\", line 224, in _create\\n return res.create(self, base_path=base_path)\\n File \\\"/usr/lib/python2.7/site-packages/openstack/resource.py\\\", line 1116, in create\\n self._translate_response(response)\\n File \\\"/usr/lib/python2.7/site-packages/openstack/resource.py\\\", line 962, in _translate_response\\n exceptions.raise_from_response(response, error_message=error_message)\\n File \\\"/usr/lib/python2.7/site-packages/openstack/exceptions.py\\\", line 229, in raise_from_response\\n http_status=http_status, request_id=request_id\\nopenstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: http://192.168.24.1:9696/v2.0/networks, Invalid input for operation: Requested MTU is too big, maximum is 1500.\\n\", \"deploy_status_code\": 1}",
                "",
                "[2019-04-28 09:15:30,371] (heat-config) [DEBUG] [2019-04-28 09:15:26,093] (heat-config) [INFO] config={\"cloud_name\": \"undercloud\", \"enable_routed_networks\": false, \"home_dir\": \"/home/stack\", \"local_ip\": \"192.168.24.1\", \"local_subnet\": \"ctlplane-subnet\", \"mtu\": 9000, \"nameservers\": [], \"physical_network\": \"ctlplane\", \"subnets\": {\"ctlplane-subnet\": {\"AllocationPools\": [{\"end\": \"192.168.24.24\", \"start\": \"192.168.24.5\"}], \"HostRoutes\": [], \"NetworkCidr\": \"192.168.24.0/24\", \"NetworkGateway\": \"192.168.24.1\"}}}",
                "[2019-04-28 09:15:26,093] (heat-config) [INFO] deploy_server_id=d0439f15-01fc-4dfe-b2d8-3b555d9bf762",
                "[2019-04-28 09:15:26,094] (heat-config) [INFO] deploy_action=CREATE",
                "[2019-04-28 09:15:26,094] (heat-config) [INFO] deploy_stack_id=undercloud-AllNodesDeploySteps-4pdudcc2fsk4-UndercloudExtraConfigPost-e7aicucc7lfp-UndercloudCtlplaneNetworkDeployment-7bqco3ns3ft5-0-wckqbj7lle2n/e35a04b9-5005-4267-b155-ed04c11b71e2",
                "[2019-04-28 09:15:26,094] (heat-config) [INFO] deploy_resource_name=TripleOSoftwareDeployment",
                "[2019-04-28 09:15:26,094] (heat-config) [INFO] deploy_signal_transport=NO_SIGNAL",
                "[2019-04-28 09:15:26,095] (heat-config) [DEBUG] Running /var/lib/heat-config/heat-config-script/f6d13638-0cff-48d1-ab4b-d89e714ef153",
                "[2019-04-28 09:15:30,361] (heat-config) [INFO] ERROR: Network create/update failed.",
                "",
                "[2019-04-28 09:15:30,361] (heat-config) [DEBUG] Traceback (most recent call last):",
                " File \"/var/lib/heat-config/heat-config-script/f6d13638-0cff-48d1-ab4b-d89e714ef153\", line 283, in <module>",
                " network = _ensure_neutron_network(sdk)",
                " File \"/var/lib/heat-config/heat-config-script/f6d13638-0cff-48d1-ab4b-d89e714ef153\", line 50, in _ensure_neutron_network",
                " mtu=CONF['mtu'])",
                " File \"/usr/lib/python2.7/site-packages/openstack/network/v2/_proxy.py\", line 1204, in create_network",
                " return self._create(_network.Network, **attrs)",
                " File \"/usr/lib/python2.7/site-packages/openstack/proxy.py\", line 224, in _create",
                " return res.create(self, base_path=base_path)",
                " File \"/usr/lib/python2.7/site-packages/openstack/resource.py\", line 1116, in create",
                " self._translate_response(response)",
                " File \"/usr/lib/python2.7/site-packages/openstack/resource.py\", line 962, in _translate_response",
                " exceptions.raise_from_response(response, error_message=error_message)",
                " File \"/usr/lib/python2.7/site-packages/openstack/exceptions.py\", line 229, in raise_from_response",
                " http_status=http_status, request_id=request_id",
                "openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: http://192.168.24.1:9696/v2.0/networks, Invalid input for operation: Requested MTU is too big, maximum is 1500.",
                "",
                "[2019-04-28 09:15:30,361] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-script/f6d13638-0cff-48d1-ab4b-d89e714ef153. [1]",
                "",
                "",
                "[2019-04-28 09:15:30,371] (heat-config) [INFO] Completed /usr/libexec/heat-config/hooks/script",
                "[2019-04-28 09:15:30,372] (heat-config) [DEBUG] Running heat-config-notify /var/lib/heat-config/deployed/f6d13638-0cff-48d1-ab4b-d89e714ef153.json < /var/lib/heat-config/deployed/f6d13638-0cff-48d1-ab4b-d89e714ef153.notify.json",
                "[2019-04-28 09:15:31,498] (heat-config) [INFO] ",
                "[2019-04-28 09:15:31,499] (heat-config) [DEBUG] "
            ]
        },
        {
            "status_code": "1"
        }
    ]
}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
cloudx-uc3 : ok=303 changed=152 unreachable=0 failed=1

Exception: Deployment failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1256, in _standalone_deploy
    raise exceptions.DeploymentError('Deployment failed')
DeploymentError: Deployment failed
None
Install artifact is located at /home/stack/undercloud-install-20190428091532.tar.bzip2

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Deployment Failed!

ERROR: Heat log files: /var/log/heat-launcher/undercloud_deploy-aOE0Ot

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Deployment failed.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

An error has occured while deploying the Undercloud.

See the previous output for details about what went wrong.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=localdomain', '--local-ip=192.168.24.1/24', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/no-tls-endpoints-public-ip.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '--cleanup', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/tripleo-validations.yaml', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 30, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/undercloud.py", line 141, in take_action
    raise exceptions.DeploymentError(e)
DeploymentError: Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=localdomain', '--local-ip=192.168.24.1/24', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/no-tls-endpoints-public-ip.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '--cleanup', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/tripleo-validations.yaml', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1
Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=localdomain', '--local-ip=192.168.24.1/24', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/no-tls-endpoints-public-ip.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '--cleanup', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/tripleo-validations.yaml', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1

Noam Angel (noama)
description: updated
Noam Angel (noama)
tags: added: mtu
Changed in tripleo:
status: New → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.opendev.org/662863

Changed in tripleo:
assignee: nobody → Alex Schultz (alex-schultz)
status: Triaged → In Progress
Changed in tripleo:
importance: Undecided → High
milestone: none → train-1
tags: added: stein-backport-potential
tags: added: rocky-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/662863
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=d6b08579abbe27e2cd58ef83f95a7a1a115c9c96
Submitter: Zuul
Branch: master

commit d6b08579abbe27e2cd58ef83f95a7a1a115c9c96
Author: Alex Schultz <email address hidden>
Date: Mon Jun 3 15:37:04 2019 -0600

    Set TenantNetPhysnetMtu to 0 to allow different MTUs

    I5e60d52ad571e1cdb3b82cd1d9947e33fa682bf8 changed the default value that
    was being used to honor the TenantNetPhysnetMtu if the
    NeutronGlobalPhysnetMtu is set to 0 (the default). For the undercloud
    we want this value to be 0 so we can define overcloud networks with
    >1500 MTU even if the undercloud is set to 1500.

    Change-Id: Ida2011b2ad0858d479b2c7d7988e1b5b908f4b3e
    Closes-Bug: #1826729

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/663047

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/stein)

Reviewed: https://review.opendev.org/663047
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=a199e335b61806178f0f192138abae87336803dc
Submitter: Zuul
Branch: stable/stein

commit a199e335b61806178f0f192138abae87336803dc
Author: Alex Schultz <email address hidden>
Date: Mon Jun 3 15:37:04 2019 -0600

    Set TenantNetPhysnetMtu to 0 to allow different MTUs

    I5e60d52ad571e1cdb3b82cd1d9947e33fa682bf8 changed the default value that
    was being used to honor the TenantNetPhysnetMtu if the
    NeutronGlobalPhysnetMtu is set to 0 (the default). For the undercloud
    we want this value to be 0 so we can define overcloud networks with
    >1500 MTU even if the undercloud is set to 1500.

    Change-Id: Ida2011b2ad0858d479b2c7d7988e1b5b908f4b3e
    Closes-Bug: #1826729
    (cherry picked from commit d6b08579abbe27e2cd58ef83f95a7a1a115c9c96)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/663400

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/rocky)

Reviewed: https://review.opendev.org/663400
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=42695fbd95bbbcc247886e62e971dd6eb786179d
Submitter: Zuul
Branch: stable/rocky

commit 42695fbd95bbbcc247886e62e971dd6eb786179d
Author: Alex Schultz <email address hidden>
Date: Mon Jun 3 15:37:04 2019 -0600

    Set TenantNetPhysnetMtu to 0 to allow different MTUs

    I5e60d52ad571e1cdb3b82cd1d9947e33fa682bf8 changed the default value that
    was being used to honor the TenantNetPhysnetMtu if the
    NeutronGlobalPhysnetMtu is set to 0 (the default). For the undercloud
    we want this value to be 0 so we can define overcloud networks with
    >1500 MTU even if the undercloud is set to 1500.

    Change-Id: Ida2011b2ad0858d479b2c7d7988e1b5b908f4b3e
    Closes-Bug: #1826729
    (cherry picked from commit d6b08579abbe27e2cd58ef83f95a7a1a115c9c96)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.0.0

This issue was fixed in the openstack/tripleo-heat-templates 11.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 9.4.0

This issue was fixed in the openstack/tripleo-heat-templates 9.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 10.6.0

This issue was fixed in the openstack/tripleo-heat-templates 10.6.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.