neutron

Welcome to Neutron’s documentation! in neutron

Bug #1811950 reported by Amer Hwitat on 2019-01-16

This bug report is a duplicate of: Bug #1811941: neutron routers are down. Edit Remove

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	neutron	New	Undecided	Unassigned

Bug Description

it seems that Red Hat Openstack 14 has problems implementing Virtual Bridges .. it affects Neutron service (Networking) on Openstack , maybe it's openvswitch issue too, Network interfaces on the Routers should not be DOWN state ...

This bug tracker is for errors with the documentation, use the following as a template and remove or add fields as you see fit. Convert [ ] into [x] to check boxes:

- [ ] This doc is inaccurate in this way: ______
- [ ] This is a doc addition request.
- [ ] I have a fix to the document that I can paste below including example: input and output.

If you have a troubleshooting or support issue, use the following resources:

- Ask OpenStack: http://ask.openstack.org
- The mailing list: http://lists.openstack.org
- IRC: 'openstack' channel on Freenode

-----------------------------------
Release: 14.0.0.0b2.dev52 on 2018-03-08 11:13:05
SHA: b09b8868e93aea437055c041148ccbd095c5c249
Source: https://git.openstack.org/cgit/openstack/neutron/tree/doc/source/index.rst
URL: https://docs.openstack.org/neutron/latest/?_ga=2.97532380.2136411735.1547571954-1573057398.1547470031

Tags:

Revision history for this message

Amer Hwitat (amer.hwitat) wrote on 2019-01-16:

openstack8-project-overview-neutron-bug-3.JPG Edit (102.0 KiB, image/jpeg)

Revision history for this message

Brian Haley (brian-haley) wrote on 2019-01-16:

This looks like duplicate of 1811941 ?

Revision history for this message

Amer Hwitat (amer.hwitat) wrote on 2019-01-19:

Download full text (15.5 KiB)

well when neutron crashes or Openstack Horizon crashes you need to clean up nodes ...

use Ironic baremetal service at (Openstack) CLI ....

sudo systemctl enable openstack-ironic-conductor
sudo systemctl restart openstack-ironic-conductor
sudo systemctl status openstack-ironic-conductor
openstack baremetal node clean ironic \
--clean-steps '[{"interface": "deploy", "step": "erase_devices_metadata"}]'

openstack baremetal node clean ironic \
--clean-steps '[{"interface": "deploy", "step": "erase_devices"}]'

openstack baremetal node clean ironic \
--clean-steps '[{"interface": "deploy", "step": "erase_devices_metadata"}]'

openstack baremetal node clean ironic \
--clean-steps '[{"interface": "deploy", "step": "erase_devices"}]'

export OS_PROJECT_DOMAIN_NAME=admin
export OS_IDENTITY_API_VERSION=3

well if like my case when I'm using VmWare VM to install allinone using answer file ... I added Ironic to the installation using packstack and set parameters --timeout=99999999999999999 and --debug to know what is going one ...I discovered after the use of a slow VM 5GB RAM 1 CPU 4 cores that the problem is that when you add routers and External or Internal Networks you are utilizing more the memory which makes neutron crashes and also Horizon dashboards with their messages that same can not retrieve limits and neutron is down ....

for the question I raised earlier I discovered that my first installation was not complete and nova was not running to give router computation power which resulted in router without cpu ...

after that I had a complete installation which eventually External Interface was down and Internal interface was UP (GREEN) ...

which takes me to another thing to ask about Openstack ... why its very hard and cost effective at the Technical side as IaaS solution ... but it's open source .. and I like it ...

related CLI commands:
systemctl disable NetworkManager
systemctl stop NetworkManager
systemctl disable firewalld
systemctl stop firewalld
setenforce 0

systemctl restart network
systemctl status network
########################################################################
subscription-manager list --available

subscription-manager attach --pool=

subscription-manager repos --enable=rhel-7-server-optional-rpms \
--enable=rhel-7-server-extras-rpms --enable=rhel-7-server-rh-common-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-devtools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-tools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-devtools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-tools-rpms
yum repolist enabled #enable all

subscriptiion-manager repos --enable=

sudo yum -y install yum-plugin-priorities yum-utils

well when neutron crashes or Openstack Horizon crashes you need to clean up nodes ...

use Ironic baremetal service at (Openstack) CLI ....

sudo systemctl enable openstack-ironic-conductor
sudo systemctl restart openstack-ironic-conductor
sudo systemctl status openstack-ironic-conductor
openstack baremetal node clean ironic \
    --clean-steps '[{"interface": "deploy", "step": "erase_devices_metadata"}]'

openstack baremetal node clean ironic \
    --clean-steps '[{"interface": "deploy", "step": "erase_devices"}]'

openstack baremetal node clean ironic \
    --clean-steps '[{"interface": "deploy", "step": "erase_devices_metadata"}]'

openstack baremetal node clean ironic \
    --clean-steps '[{"interface": "deploy", "step": "erase_devices"}]'

export OS_PROJECT_DOMAIN_NAME=admin
export OS_IDENTITY_API_VERSION=3

for the question I raised earlier I discovered that my first installation was not complete and nova was not running to give router computation power which resulted in router without cpu ...

after that I had a complete installation which eventually External Interface was down and Internal interface was UP (GREEN) ...

which takes me to another thing to ask about Openstack ... why its very hard and cost effective at the Technical side as IaaS solution ... but it's open source .. and I like it ...

related CLI commands:
systemctl disable NetworkManager
systemctl stop NetworkManager
systemctl disable firewalld
systemctl stop firewalld
setenforce 0

systemctl restart network
systemctl status network
########################################################################
subscription-manager list --available

subscription-manager attach --pool=

subscription-manager repos --enable=rhel-7-server-optional-rpms \
  --enable=rhel-7-server-extras-rpms --enable=rhel-7-server-rh-common-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-devtools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-14-tools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-devtools-rpms
subscription-manager repos --enable=rhel-7-server-openstack-13-tools-rpms
yum repolist enabled #enable all

subscriptiion-manager repos --enable=

sudo yum -y install yum-plugin-priorities yum-utils

yum install openstack-selinux --skip-broken
yum install python2* --skip-broken 
yum install *pyparsing* --skip-broken
yum install *urllib3* --skip-broken
yum install *chardet* --skip-broken
yum install *gnocchi* --skip-broken
yum install *puppet* --skip-broken
yum install -y mariadb-server-galera --skip-broken
yum install -y *openvswitch* --skip-broken
#yum install -y openstack.*   #saves time installing with packstack#yum install -y puppet hiera openssh-clients tar nc rubygem-json
yum install -y puppet hiera openssh-clients tar nc rubygem-json --skip-broken
yum install -y *rabbitmq* --skip-broken
yum install -y *ironic* --skip-broken
rpm -q --whatprovides rubygem-json   ###### rubygem-json-1.7.7-20.el7.x86_64

#do not update with ### yum update ### it will upgrade the RHEL 7 to 7.6 Maipo if you want to keep 7.0 on your VM
yum install -y openstack-packstack --skip-broken
packstack --answer-file=/root/anwer.txt --timeout=99999999999999 --debug ### :) time out to complete installation on a slow machine

################################################################################

export OS_USERNAME=admin
export OS_PASSWORD=server
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://localhost:5000/v3

neutron net-create External1 --provider:network_type flat --provider:physical_network br-ex --router:external=true --shared
neutron net-create External2 --provider:network_type flat --provider:physical_network eno16777736 --router:external=true --shared
neutron net-create External3 --provider-physical-network provider --provider:physical_network eno16777736 --router:external=true --shared
openstack subnet create --network provider \
  --allocation-pool start=192.168.43.2,end=192.168.43.240 \
  --dns-nameserver 192.168.43.1 --gateway 192.168.43.1 \
  --subnet-range 192.168.43.0/24 provider

mysql
create database neutron;
grant all privileges on neutron.* to 'neutron'@'localhost' identified by 'server';
grant all privileges on neutron.* to 'neutron'@'%' identified by 'server';
quit

export | grep OS_declare -x OS_AUTH_URL="https://192.168.64.128:5000/v3"

#source admin-openrc.sh
. /root/keystonerc_admin
openstack user create --domain default --password-prompt neutron
openstack role add --project service --user neutron admin
openstack service create --name neutron --description "OpenStack Networking" network
openstack endpoint create --region RegionOne network public http://controller:9696
openstack endpoint create --region RegionOne network internal http://controller:9696
openstack endpoint create --region RegionOne network admin http://controller:9696

systemctl enable neutron-server.service neutron-openvswitch-agent.service neutron-dhcp-agent.service neutron-metadata-agent.service neutron-ovs-cleanup.service
systemctl start neutron-server.service neutron-openvswitch-agent.service neutron-dhcp-agent.service neutron-metadata-agent.service neutron-ovs-cleanup.service
systemctl enable neutron-l3-agent.service
systemctl start neutron-l3-agent.service

############################################################################################
######################################################################################
Horizon admin unauthorized access error

export OS_USERNAME=admin
export OS_PASSWORD=server
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://localhost:5000/v3

export OS_USERNAME=admin
export OS_PASSWORD=server
export OS_PROJECT_NAME=project
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_DOMAIN_ID=default
export OS_AUTH_URL=http://auth.example.com:5000/v3
##############################################################################################
/etc/cinder/cinder.conf

added

[keystone_authtoken]
auth_uri = https://keystone_ip:5000
auth_url = https://keystone_ip:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = services
username = cinder
password = [ccinder password]

systemctl restart openstack-nova-api 
systemctl restart openstack-nova-cert 
systemctl restart openstack-nova-consoleauth 
systemctl restart openstack-nova-scheduler 
systemctl restart openstack-nova-conductor 
systemctl restart openstack-nova-novncproxy
systemctl restart neutron-server 
systemctl restart neutron-dhcp-agent
systemctl restart neutron-l3-agent
systemctl restart neutron-metadata-agent
systemctl restart neutron-openvswitch-agent
systemctl restart openstack-cinder-api
systemctl restart openstack-cinder-backup
systemctl restart openstack-cinder-scheduler
systemctl restart openstack-cinder-volume

sudo service --status-all | grep nova
sudo service --status-all | grep neutron
sudo service --status-all | grep cinder
##########################################################################################
to clean up after a crash

sudo systemctl enable openstack-ironic-conductor
sudo systemctl restart openstack-ironic-conductor
sudo systemctl status openstack-ironic-conductor
openstack baremetal node clean ironic \
    --clean-steps '[{"interface": "deploy", "step": "erase_devices_metadata"}]'

openstack baremetal node clean ironic \
    --clean-steps '[{"interface": "deploy", "step": "erase_devices"}]'

openstack baremetal node clean ironic \
    --clean-steps '[{"interface": "deploy", "step": "erase_devices_metadata"}]'

openstack baremetal node clean ironic \
    --clean-steps '[{"interface": "deploy", "step": "erase_devices"}]'

export OS_PROJECT_DOMAIN_NAME=admin
export OS_IDENTITY_API_VERSION=3

openstack network show External2 -f value -c id

openstack baremetal node list --provision-state available --no-maintenance --unassociated

keystone service-create --name=ironic --type=ironic --description="Orchestration"

keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ orchestration / {print $2}') \ 
--publicurl=http://controller:8004/v3/%$tenant_id$s \
--internalurl=http://controller:8004/v3/%$tenant_id$s \
--adminurl=http://controller:8004/v3/%$tenant_id$s

openstack user create --password server --enable ironicusr
openstack role add --project service --user ironicusr admin
openstack service create --name ironic --description "Ironic bare metal provisioning service" baremetal
openstack endpoint list -c "Service Name" -c "PublicURL" --long
route -n
openstack endpoint create --publicurl http://192.168.43.110:6385 --internalurl http://192.168.43.110:6385 --adminurl http://192.168.43.110:6385 --region regionOne  97effffc1f96444d94143b352e37e2fb

#######################################################################################

listen ironic

bind VIP:6385 transparent
  server SERVER_NAME COMPUTE_INTERNAL_API_IP:6385 check fall 5 inter 2000 rise 2

In this example:

VIP is the virtual IP address.
    SERVER_NAME is the HAProxy identifying name for the Compute server where the Bare Metal Provisioning service will be installed and running.
    COMPUTE_INTERNAL_API_IP is the Internal API IP address of the Compute server where the Bare Metal Provisioning service will be installed and running.
    transparent allows the HAProxy to bind the IP address even if it does not exist on the Controller node so that in a clustered environment, the virtual IP address can move between controllers.

check fall 5 inter 2000 rise 2 refers to the following health checks on the back end server:
        fall 5 - the server is considered unavailable after 5 consecutive failed health checks.
        inter 2000 - the interval between health checks is 2000 ms or 2 seconds.
        rise 2 - the server is considered available after 2 consecutive successful health checks.

Restart the HAProxy to make sure the changes take effect:

# systemctl restart haproxy.service
systemctl restart haproxy.service

Creating the Bare Metal Provisioning Database

Connect to the database service:

# mysql -u root

Create the ironic database:

mysql> CREATE DATABASE ironic CHARACTER SET utf8;

Create an ironic database user and grant the user access to the ironic database:

mysql> GRANT ALL PRIVILEGES ON ironic.* TO 'ironic'@'%' IDENTIFIED BY 'PASSWORD';
    mysql> GRANT ALL PRIVILEGES ON ironic.* TO 'ironic'@'localhost' IDENTIFIED BY 'PASSWORD';

Replace PASSWORD with a secure password that will be used to authenticate with the database server as this user.

Flush the database privileges to ensure that they take effect immediately:

mysql> FLUSH PRIVILEGES;

Exit the mysql client:

mysql> quit

openstack-config --set /etc/nova/nova.conf \
   DEFAULT scheduler_host_manager nova.scheduler.ironic_host_manager.IronicHostManager

openstack-config --set /etc/nova/nova.conf DEFAULT scheduler_tracks_instance_changes false

openstack-config --set /etc/nova/nova.conf DEFAULT baremetal_scheduler_default_filters AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter
openstack-config --set /etc/nova/nova.conf \
   ironic admin_username ironic
openstack-config --set /etc/nova/nova.conf \
   ironic admin_password server
openstack-config --set /etc/nova/nova.conf \
   ironic admin_url http://192.168.43.110:35357/v3
openstack-config --set /etc/nova/nova.conf \
   ironic admin service
openstack-config --set /etc/nova/nova.conf \
   ironic api_endpoint http://192.168.43.110:6385/v1

openstack-config --set /etc/nova/nova.conf database connection "mysql+pymysql://nova:server@DB_IP/nova"
systemctl restart openstack-nova-scheduler.service
systemctl restart openstack-nova-compute.service

openstack endpoint create --region RegionOne \
--publicurl http://192.168.43.110:6385 \
--internalurl http://192.168.43.110:6385 \
--adminurl http://192.168.43.110:6385 \
baremetal

export OS_AUTH_TYPE=server
###############################################################################################

well bottom line it doesn't seems to be a bug but it's more of a slow VM issue with neutron and nova whereas services goes down and malfunction on slow machines .....

Like this Error when I was configuring Br-Ex (Bridge)
 
[root@localhost network-scripts]# systemctl status network -l
● network.service - LSB: Bring up/down networking
   Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 2019-01-19 03:47:01 EST; 21s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 86319 ExecStop=/etc/rc.d/init.d/network stop (code=exited, status=0/SUCCESS)
  Process: 86591 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=1/FAILURE)
    Tasks: 0

Jan 19 03:47:01 localhost.localdomain dhclient[86963]: Please report for this software via the Red Hat Bugzilla site:
Jan 19 03:47:01 localhost.localdomain dhclient[86963]:     http://bugzilla.redhat.com
Jan 19 03:47:01 localhost.localdomain dhclient[86963]: ution.
Jan 19 03:47:01 localhost.localdomain dhclient[86963]: exiting.
Jan 19 03:47:01 localhost.localdomain network[86591]: failed.
Jan 19 03:47:01 localhost.localdomain network[86591]: [FAILED]
Jan 19 03:47:01 localhost.localdomain systemd[1]: network.service: control process exited, code=exited status=1
Jan 19 03:47:01 localhost.localdomain systemd[1]: Failed to start LSB: Bring up/down networking.
Jan 19 03:47:01 localhost.localdomain systemd[1]: Unit network.service entered failed state.
Jan 19 03:47:01 localhost.localdomain systemd[1]: network.service failed.
[root@localhost network-scripts]#

I discovered that there is no Bug it's another ifcfg-eno16777736_1 file in /etc/sysconfig/network-scripts that the system think it's the same NIC card ...

well thanks Brian for the support ... you are the only one who replied and takes the credit of my Best regards

Report a bug

This report contains Public information

Everyone can see this information.

Duplicate of bug #1811941 Remove

You are

Subscribing...

Edit bug mail

Other bug subscribers

Bug attachments

openstack8-project-overview-neutron-bug-3.JPG Edit

Add attachment

Remote bug watches

Bug watches keep track of this bug in other bug trackers.