feature request: have option to use ipv4 when setting up network configuration for octavia
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Octavia Charm |
Opinion
|
Undecided
|
Unassigned |
Bug Description
By default, Octavia charm uses ipv6 for its lb-mgmt-subnet.[1]
It would be nice to have the option to choose an ipv4 network from the start instead of deleting the ipv6 network and recreating the ipv4 subnet.
Implementation - possible configuration option parameter when deploying.
Dan MacDonald (allcoms) wrote : | #1 |
Heather Lemon (hypothetical-lemon) wrote : | #2 |
I personally have not tested the Octavia Charm with it's default IPv6 network setup.
I can elaborate on creating an IPv4 network with Octavia.
So by default the Octavia charm uses IPV6. We will need to delete the IPv6 and add an ipv4 network.
# delete any ipv6 related network with octavia
openstack network list
openstack subnet list
openstakc network show lb-mgmt
openstack router remove subnet lb-mgmt lb-mgmt-subnetv6
openstack subnet delete lb-mgmt-subnetv6
#create ipv4 octavia network, subnet, router, add subnet to router
openstack network create lb-mgmt-net --tag charm-octavia
openstack subnet create --tag charm-octavia --subnet-range 10.0.0.3/24 --dhcp --ip-version 4 --network lb-mgmt-net lb-mgmt-subnet
openstack router create lb-mgmt --tag charm-octavia
openstack router add subnet lb-mgmt lb-mgmt-subnet
#add security rules
openstack security group create lb-mgmt-sec-grp --tag charm-octavia
openstack security group create lb-health-
openstack security group rule create lb-mgmt-sec-grp --protocol icmp
openstack security group rule create lb-mgmt-sec-grp --protocol tcp --protocol tcp --dst-port 22
openstack security group rule create lb-mgmt-sec-grp --protocol tcp --dst-port 9443
#create loadbalancer
openstack loadbalancer create --name lb0 --vip-network-id lb-mgmt
# You may or may not need to do this part.
# create temp novarc.services and extract octavia password
touch /tmp/novarc.
cat << EOF > /tmp/novarc.
OS_PROJECT_
OS_USERNAME=octavia
OS_PROJECT_
OS_USER_
OS_PASSWORD=$(juju run --unit octavia/0 "grep -v "auth" /etc/octavia/
EOF
source /tmp/novarc.
I had issues where when creating the loadbalancers it would be creating them from the admin area and not the services area so it would fail.
Dan MacDonald (allcoms) wrote : | #3 |
Thanks v.much Heather! That's very helpful. I'm going to be trying this next week.
Frode Nordahl (fnordahl) wrote : | #4 |
The charm already allows you to manage the cloud resources manually by setting the `create-
If you have issues with the tunneled communication from the Octavia units to the cloud the root cause of that is most likely something else than the address family being used inside the tunnel.
0: https:/
1: https:/
2: https:/
Changed in charm-octavia: | |
status: | New → Opinion |
Dan MacDonald (allcoms) wrote : | #5 |
We are using bind on our MAAS server as our DNS server.
If I have the line:
listen-on-v6 { any; };
Uncommented in /etc/bind/
Dan MacDonald (allcoms) wrote : | #6 |
Our OS install is out of action so I can't check at the moment but did you have to create the service_domain domain with an octavia user in it or does the Octavia charm do that? I presume I can assign it to any domain/
I presume multiple projects/users can use Octavia?
Frode Nordahl (fnordahl) wrote : | #7 |
The internal communication between the Octavia worker and its amphorae has nothing to do with your physical network, MAAS or Juju configuration. The octavia units participate in the overlay network in your OpenStack cloud and the internal communication is tunneled through that.
Heather Lemon (hypothetical-lemon) wrote : | #8 |
From what I remember the octavia user should be created by the charm
| I presume multiple projects/users can use Octavia?
Do you mean the use the Octavia user? Or just Octavia in general?
In general yeah multiple projects/users can use octavia
Dan MacDonald (allcoms) wrote : | #9 |
Hi Heather and Frode
I have now redeployed the OS bionic-train telemetry bundle but this time I set
create-
and then I've used exactly the same settings as you to create an IPv4 Octavia management network but I've not been able to successfully create a LB yet.
I'm pretty sure your create a lb command above was supposed to be:
openstack loadbalancer create --name lb0 --vip-network-id lb-mgmt-net
That command errors out pretty quickly for me, as does:
lb_vip_
I am running these commands when authenticated as the master admin OS user. I am presuming there is no need to add any octavia policy roles to the admin user to create lbs is there?
I've not tried sourcing your novarc.services yet because I'm unsure how you intend me to use that? Most of its vars clash with those required for me to use OS. Do you source that before creating lbs? Do your source it after you source your regular OS users credentials?
In /var/log/
2020-12-17 21:03:38.830 215189 ERROR oslo_messaging.
Those security groups that it says don't exist do.
$ openstack security group list
+------
| ID | Name | Description | Project | Tags |
+------
| 065f6c3c-
| 07bd80a2-
| 7486748f-
| 89600e62-
Dan MacDonald (allcoms) wrote : | #10 |
"I had issues where when creating the loadbalancers it would be creating them from the admin area and not the services area so it would fail."
When you say "admin area", what do you mean? The admin project or the admin internal network / subnet?
Dan MacDonald (allcoms) wrote : | #11 |
Which versions of OS and Ubuntu are you using Heather?
Heather Lemon (hypothetical-lemon) wrote : | #12 |
Do you source that before creating lbs? Do your source it after you source your regular OS users credentials?
Yes I do source novarc.services before creating the loadbalancers.
by "admin area" i mean the admin network.
This is your problem:
I am running these commands when authenticated as the master admin OS user.
You need to be creating the loadbalancer with an octavia user
This error:
Last exception: Security group 7486748f-
Tells you that the octavia network isn't being found/used by the "services" project
can you run `openstack project list` and do you see one called services? cause that's where all of the networking and configuration for octavia needs to come from, not the admin anything
What does your octavia security groups look like?
Here's an example to setup security groups with the "services"
----
openstack security group rule add 4b965a7d-
openstack security group rule create 4b965a7d-
openstack security group rule add 4b965a7d-
openstack security group rule create 4b965a7d-
openstack security group rule create lb-mgmt-sec-grp --protocol tcp --protocol tcp --dst-port 22 --project services
juju ssh octavia/0 sudo -s
openstack server list --all-projects
openstack security group rule create 350af0bc-
openstack security group rule create e5847ca9-
openstack security group rule list 350af0bc-
openstack security group rule create 350af0bc-
juju ssh octavia/0 sudo reboot
juju ssh octavia/0 ip a s o-hm0
distro info
----
cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_
DISTRIB_
DISTRIB_
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https:/
SUPPORT_URL="https:/
BUG_REPORT_URL="https:/
PRIVACY_
VERSION_
UBUNTU_
---
hope this helps
Have a happy holidays :)
Dan MacDonald (allcoms) wrote : | #13 |
Thanks for that info
After installing the telemetry bundle and telling the octavia charm not to create any networks, I have 2 projects called services and 2 called admin.
How to know which is the correct one to add the security rules to? I could add the rules to both but I'd like to know which is the one octave will use.
$ openstack project list
+------
| ID | Name |
+------
| 467ce649c1e2482
| 8138d057750d4e0
| 9dc00db9733049c
| 9dcc7bbb0d93493
+------
$ openstack project show 467ce649c1e2482
+------
| Field | Value |
+------
| description | Created by Juju |
| domain_id | default |
| enabled | True |
| id | 467ce649c1e2482
| is_domain | False |
| name | services |
| options | {} |
| parent_id | default |
| tags | [] |
+------
$ openstack project show 9dcc7bbb0d93493
+------
| Field | Value |
+------
| description | Created by Juju |
| domain_id | 8d54cf927e0f40b
| enabled | True |
| id | 9dcc7bbb0d93493
| is_domain | False |
| name | services |
| options | {} |
| parent_id | 8d54cf927e0f40b
| tags | [] |
+------
juju ssh octavia/0 ip a s o-hm0
I should have a network interface on octavia called o-hm0?
Heather Lemon (hypothetical-lemon) wrote : | #14 |
| should have a network interface on octavia called o-hm0?
- No, you don't need a network interface called o-hm0. that was a typo on my part, it's just for confirmation that an ipv4 network address was being used by octavia.
Hmm... I only have one network and one services project.
Heather Lemon (hypothetical-lemon) wrote : | #15 |
Could you run `openstack role assignment list --names`
this tells us which projects and roles are assigned
Dan MacDonald (allcoms) wrote : | #16 |
Keeping in mind that I have since tried to run:
juju run-action --wait octavia/0 configure-resources
after briefly setting:
juju config octavia create-
but, to my total lack of surprise, configure-resources failed to complete without the ipv4 networks being pre-prepared so I am back to working on getting Octavia working with IPv4.
I noticed that juju doesn't seem too aware of the octavia management networks because I deleted these ipv4 ones to try ipv6 again but juju still claimed octavia was active.
So, this output is likely to some roles created by an unsuccessful run of configure-resources
$ openstack role assignment list --names
+------
| Role | User | Group | Project | Domain | System | Inherited |
+------
| Admin | cinderv2_
| Admin | neutron@
| Admin | octavia@Default | | services@Default | | | False |
| load-balancer_
| load-balancer_admin | octavia@Default | | services@Default | | | False |
| load-balancer_
| load-balancer_
| load-balancer_
| Admin | gnocchi@
| Admin | admin@admin_domain | | admin@admin_domain | | | False |
| member | admin@admin_domain | | admin@admin_domain | | | False |
| Admin | admin@admin_domain | | | admin_domain | | False |
| Admin | nova@Default | | services@Default | | | False |
| Admin | ceilometer@
| ResellerAdmin ...
Heather Lemon (hypothetical-lemon) wrote : | #17 |
Yeah i actually got the same error. My coworker is currently looking into why configure-resources is failing
Heather Lemon (hypothetical-lemon) wrote : | #18 |
Heather Lemon (hypothetical-lemon) wrote : | #19 |
related configure-resources error message failure
+ juju run-action octavia/leader configure-resources --wait
unit-octavia-0:
UnitId: octavia/0
id: "2"
message: cannot unpack non-iterable NoneType object
results: {}
status: failed
timing:
completed: 2020-12-18 17:35:32 +0000 UTC
enqueued: 2020-12-18 17:35:05 +0000 UTC
started: 2020-12-18 17:35:26 +0000 UTC
Dan MacDonald (allcoms) wrote : | #20 |
I'm still scratching my head over this and wondering how anybody has ever got Octavia to work.
I'm still uncertain as to which of my two services projects is the relevant one. I tried looking at the output of various versions of `openstack role assignment list` (eg listing by project etc) but there were no give aways.
I have several questions regarding your novarc.services file which you say you need to run before creating LBs. You are the only person / guide / resource who has mentioned the need for a creating specific RC file for these purposes. Which guide or document contains this info? There is no mention of having to create such an rc file here:
https:/
Nor is any such file mentioned in the docs for the Octavia charm. Additionally, there are a few issues with it so I'm wondering if you copy/pasted an old (unworking) version? The command to fetch the Octavia password with juju doesn't work but if I replace that with my octavia password then source it I get:
Missing value auth-url required for auth plugin password
So, I've tried adding
export OS_AUTH_
I'm not really impressed with
https:/
as its overly vague for my liking. It starts like this:
"Create Octavia User
By default Octavia will use the ‘octavia’ user for keystone authentication, and the admin user for interactions with all other services.
You must:
Create ‘octavia’ user."
Create octavia user where? How? What sort of user (Openstack? Linux? Octavia? Keystone)? With what command? The rest of the document isn't any better.
Dan MacDonald (allcoms) wrote : | #21 |
I've tried using:
export OS_AUTH_URL=
Using the URL for Keystone and the URL for the Octavia unit in the novarc.services with no luck.
Dan MacDonald (allcoms) wrote : | #22 |
Here are extracts of the relevant parts of my notes on deploying OpenStack so far. If you can spot any issues or if you have similar notes you could share I'd appreciate your feedback:
First I had to make a few small edits to the bundle.yaml so that it would deploy correctly, then:
Presuming you are currently in the directory of the correct bundle, you can deploy it with the Octava overlay by running:
$ juju deploy ./bundle.yaml --overlay ../overlays/
Deploying the Openstack telemetry bundle usually takes just under 40 minutes on our current hardware. Keystone and nova-cloud-
Get login details
$ source openrc
$ env | grep OS_
$ juju status openstack-dashboard
Make a note somewhere secure of OS_PASSWORD and the IP address of the dashboard.
Download an amd64 Ubuntu cloud installation image
$ curl https:/
openstack image create --public --container-
Create an external network
A shared external network must be defined in addition to every user/project having their own private network, usually created by the new user script
$ openstack network create --external --provider-
--provider-
$ openstack subnet create --subnet-range 172.20.0.0/21 --no-dhcp \
--gateway 172.20.0.50 --network ext_net \
--allocatio
The address passed to the gateway option is the address of the MAAS controller.
Create an internal network for the admin user
$ openstack network create internal
$ openstack subnet create --network internal \
--subnet-range 198.51.100.0/24 \
--dns-
internal_subnet
$ openstack router create provider-router
$ openstack router set --external-gateway ext_net provider-router
$ openstack router add subnet provider-router internal_subnet
Create a flavor
$ openstack flavor create --ram 2048 --disk 20 m1.small
Create security group rules to enable SSH and ping
$ PROJECT_
--domain admin_domain)
$ SECGRP_
| awk '/default/
$ openstack security group rule create $SECGRP_ID \
--protocol icmp --ingress --ethertype IPv4
$ openstack security group rule create $SECGRP_ID \
--protocol icmp --ingress --ethertype IPv6
$ openstack security group rule create $SECGRP_ID \
--protocol tcp --ingress --ethertype IPv4 --dst-port 22
$ openstack security group rule create $SECGRP_ID \
--protocol tcp --ingress --ethertype IPv6 --dst-port 22
Fixing Ceilometer and Ceph
If you get the error Services not running that should be: haproxy for ceph-radosgw/0, simply SSH into it and...
Dan MacDonald (allcoms) wrote : | #23 |
I shouldn't have cut this bit out:
Edit ~/openstack-
create-
Hence I didn't need to delete any networks.
Heather Lemon (hypothetical-lemon) wrote : | #24 |
Hi Dan,
I hope you had a relaxing holiday.
Yes setting the,
| create-
will help out a lot and not having to delete any networks.
One of my coworkers will be better to answer your questions about setting up the Octavia user, he's still on break, but will be back soon.
Dan MacDonald (allcoms) wrote : | #25 |
Hi Heather
I had a good holiday thanks. Happy new year and big thanks to you an your co-worker for helping me get Octavia to work!
I just wanted to clarify the novarc.services file content. The first thing I tried was what you pasted earlier which amounts to:
OS_PROJECT_
OS_USERNAME=octavia
OS_PROJECT_
OS_USER_
OS_PASSWORD=$(juju run --unit octavia/0 "grep -v "auth" /etc/octavia/
My guess would be it should look something more like:
export OS_AUTH_URL=http://
export OS_PROJECT_
export OS_USERNAME=octavia
export OS_PROJECT_
export OS_PROJECT_
export OS_USER_
export OS_PASSWORD=XXX
Where OS_AUTH_URL is either the address of Keystone or maybe the Octavia server, OS_PASSWORD is the auth password contained within /etc/octavia/
Dan MacDonald (allcoms) wrote : | #26 |
I was looking at the wrong installation guide for Octavia. This one has more details and does mention creating a rc file for Octavia:
I'm still uncertain how to identify the correct services project to use when there are multiple.
Heather Lemon (hypothetical-lemon) wrote : | #27 |
Yeah, I'm not sure on how to identify more than one services project when multiple are being used. Let me ask around and get back to you.
Heather Lemon (hypothetical-lemon) wrote : | #28 |
What does this command look like?
openstack role assignment list
Heather Lemon (hypothetical-lemon) wrote : | #29 |
openstack role assignment list --names
Dan MacDonald (allcoms) wrote : | #30 |
Just to add that I got this working eventually using the charm default (ipv6) network but using ipv4 everywhere else.
Heather Lemon (hypothetical-lemon) wrote : | #31 |
Oh man thats great!
Ah okay interesting, thanks for the update.
If you give this a try tho, openstack role assignment list --names
it should help you sort out where projects are coming from.
A coworker mentioned this too:
create a user with the member role and all the octavia roles (in the default domain) in its own project and in the default domain - then try see if setting up a load balancer as this user
Thanks,
Heather Lemon
I am currently trying to get Octavia to work. Does Octavia not work for you when you try to use the default ipv6 subnet?
Could you please explain how you create an IPv4 subnet for Octavia?
Thanks