openstack-manuals

Placement endpoint configuration incorrect as written

Bug #1683585 reported by Darren McGrandle on 2017-04-18

This bug affects 7 people

Affects		Status	Importance	Assigned to	Milestone
	openstack-manuals	Fix Released	High	Brian Moss

Bug Description

Installed on vanilla ubuntu 16.04 LTS, using ocata cloud repo.

- [X] This doc is inaccurate in this way: ______

The guide indicates that endpoints should be created without specifying the port number, as so:

openstack endpoint create --region RegionOne placement public http://controller/placement
openstack endpoint create --region RegionOne placement admin http://controller/placement
openstack endpoint create --region RegionOne placement internal http://controller/placement

however, this does not work, and produces error messages in the nova logs of the compute nodes containing messages like:

ERROR nova.scheduler.client.report [req-7aec8f56-4d2d-4e6f-8cb8-10af0aa06c99 - -] Failed to create resource provider record in placement API for UUID 04b70101-7ec5-4078-b500-203c5560abf9. Got 404: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
  <html><head>
  <title>404 Not Found</title>
  </head><body>
  <h1>Not Found</h1>
  <p>The requested URL /placement/resource_providers was not found on this server.</p>
  <hr>
  <address>Apache/2.4.18 (Ubuntu) Server at 10.23.124.235 Port 80</address>
  </body></html>

- [ ] This is a doc addition request.
- [X] I have a fix to the document that I can paste below including example: input and output.

Looking at /etc/apache2/sites-enabled/nova-placement-api.conf, I could clearly see that the placement api is not on port 80, but rather on port 8778. To fix, I went back and deleted the incorrect endpoints and re-created them like so:

openstack endpoint create --region RegionOne placement public http://controller:8778
openstack endpoint create --region RegionOne placement admin http://controller:8778
openstack endpoint create --region RegionOne placement internal http://controller:8778

After restarting the services, now the compute node could successfully register data with the controller and I could successfully launch instances.

-----------------------------------
Release: 15.0.0 on 2017-04-14 06:14
SHA: c73899569e19fc809f56f6d58235f339c30bda2a
Source: https://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/install-guide/source/nova-controller-install.rst
URL: https://docs.openstack.org/ocata/install-guide-ubuntu/nova-controller-install.html

Tags:

Revision history for this message

Alexandra Settle (alexandra-settle) wrote on 2017-04-19:

Adding Brian to the report as he initially made this change.

Brian - any insight?

Revision history for this message

Brian Moss (bmoss) wrote on 2017-04-20:

Feedback on the original patch was that the use of ports was discouraged for endpoints and we should just use /placement instead. But I'll run through an install next week and see what happens.

Changed in openstack-manuals:
assignee:	nobody → Brian Moss (bmoss)

Revision history for this message

Alexandra Settle (alexandra-settle) wrote on 2017-04-20:

Brian - another bug reported (that I just marked as duplicate) with this same issue. Thanks for looking into this!

Revision history for this message

Alexandra Settle (alexandra-settle) wrote on 2017-04-20:

https://bugs.launchpad.net/openstack-manuals/+bug/1684115

Revision history for this message

Chris Dent (cdent) wrote on 2017-04-21:

I think the problem stems from the fact that I commented in the docs that http://controller/placement is the desired location(as part of a long term plan to use prefixes not ports for the various services). However, the Ubuntu and RDO packages don't reflect that and the docs need to use the URLs of where the packagers set up the service.

We need to locate the corresponding people from the packagers to find out what there preference is. That's in progress.

Alexandra Settle (alexandra-settle) on 2017-04-21

Changed in openstack-manuals:
status:	New → Confirmed
importance:	Undecided → High

Revision history for this message

Brian Moss (bmoss) wrote on 2017-04-24:

Chris, in the short term should I change this back to port numbers then? I believe that currently works for all distros.

Revision history for this message

Chris Dent (cdent) wrote on 2017-04-24:

Brian, I think it depends on the current state of the packages. I don't have any clear insight into that.

Revision history for this message

Matt Riedemann (mriedem) wrote on 2017-04-24:

We should ask Corey Bryant since he knows what's going on with the ubuntu packages (assuming he's still with Canonical).

tags:

added: install-guide nova placement

Revision history for this message

Tylar (murray-tylar) wrote on 2017-04-24:

I was just working through the RDO docs and have been snagged by this issue for a few days now.

The suggested fix of changing the endpoints worked for me (although I used `openstack endpoint set --url http://controller:8778 $endpointID`).

The relevant file for RDO `/etc/httpd/conf.d/00-nova-placement-api.conf` defaulted to listening on port `8778` for me as well.

Revision history for this message

Corey Bryant (corey.bryant) wrote on 2017-04-24:

#10

I think this should be moved back to using port numbers for now.

Revision history for this message

Matt Riedemann (mriedem) wrote on 2017-04-24:

#11

Here is the IRC conversation we had in #openstack-nova today:

(2:21:23 PM) mriedem: coreycb: can you comment on the state of the nova-placement-api package wrt https://bugs.launchpad.net/openstack-manuals/+bug/1683585 /
(2:21:24 PM) mriedem: ?
(2:21:25 PM) openstack: Launchpad bug 1683585 in openstack-manuals "Placement endpoint configuration incorrect as written" [High,Confirmed] - Assigned to Brian Moss (bmoss)
coreycb coreywright
(2:21:54 PM) coreycb: mriedem, sure, taking a look
(2:22:15 PM) mriedem: smcginnis: sounds like you should be able to do the workaround in that bug report by deleting your endpoints and recreating them with the 8778 port
(2:22:33 PM) smcginnis: mriedem: Just finished recreating them, let's see how this goes.
(2:22:41 PM) ***mriedem crosses fingers
(2:24:22 PM) coreycb: smcginnis, i've always created them with the port numbers, i need to what chris was talking about
(2:24:35 PM) coreycb: that is, i need to look into
(2:25:20 PM) cdent: coreycb: I think the fundamental issue was that I said (in a review of the install guide): in the long term we want to use prefixes like /placement instead of ports
(2:25:26 PM) cdent: but that got translated into changing the docs
(2:25:36 PM) smcginnis: coreycb: Great. Yeah, I was just following the docs.o.o ubuntu instructions, and that doesn't work right now.
(2:25:37 PM) cdent: causing a mismatch with the packges
(2:26:02 PM) smcginnis: Hmm, not I see this in the scheduler log: The placement API endpoint not found. Placement is optional in Newton, but required in Ocata. Please enable the placement service before upgrading.
(2:26:37 PM) mriedem: smcginnis: you're getting an endpoint not found error from ksa now
(2:26:59 PM) mriedem: smcginnis: the service type and endpoint name are 'placement' right?
(2:27:11 PM) coreycb: cdent, yep
(2:27:55 PM) mriedem: cdent: unless the ubuntu packages are going to change to take the port number out of the apache config it seems we should get the install guide updated to create the endpoint with the port number
(2:28:07 PM) cdent: mriedem: yes
(2:28:15 PM) coreycb: mriedem, cdent: yes agreed
(2:28:25 PM) cdent: that's pretty much where asettle and I landed when talking about it last week
(2:28:30 PM) mriedem: if canonical changed the apache config in a new package release, that would be confusing (at least for ocata)
(2:28:50 PM) mriedem: you could probably mirate the config somehow at install/update time for the new package, but...
(2:28:54 PM) mriedem: *migrate
(2:29:01 PM) smcginnis: Well, if our long term goal is to get rid of ports, shouldn't we make sure new services aren't getting set up that way by default?
(2:29:36 PM) mriedem: smcginnis: sure, but then canonical has to change the config in their package and provide a migration i'd think
(2:29:43 PM) mriedem: which is probably a PITA after ocata GA'ed
(2:29:58 PM) coreycb: mriedem, smcginnis: yeah we'd definitely want to do that on a release boundary if we did

Here is the IRC conversation we had in #openstack-nova today:

(2:21:23 PM) mriedem: coreycb: can you comment on the state of the nova-placement-api package wrt https://bugs.launchpad.net/openstack-manuals/+bug/1683585 /
(2:21:24 PM) mriedem: ?
(2:21:25 PM) openstack: Launchpad bug 1683585 in openstack-manuals "Placement endpoint configuration incorrect as written" [High,Confirmed] - Assigned to Brian Moss (bmoss)
coreycb coreywright 
(2:21:54 PM) coreycb: mriedem, sure, taking a look
(2:22:15 PM) mriedem: smcginnis: sounds like you should be able to do the workaround in that bug report by deleting your endpoints and recreating them with the 8778 port
(2:22:33 PM) smcginnis: mriedem: Just finished recreating them, let's see how this goes.
(2:22:41 PM) ***mriedem crosses fingers
(2:24:22 PM) coreycb: smcginnis, i've always created them with the port numbers, i need to what chris was talking about
(2:24:35 PM) coreycb: that is, i need to look into
(2:25:20 PM) cdent: coreycb: I think the fundamental issue was that I said (in a review of the install guide): in the long term we want to use prefixes like /placement instead of ports
(2:25:26 PM) cdent: but that got translated into changing the docs
(2:25:36 PM) smcginnis: coreycb: Great. Yeah, I was just following the docs.o.o ubuntu instructions, and that doesn't work right now.
(2:25:37 PM) cdent: causing a mismatch with the packges
(2:26:02 PM) smcginnis: Hmm, not I see this in the scheduler log: The placement API endpoint not found. Placement is optional in Newton, but required in Ocata. Please enable the placement service before upgrading.
(2:26:37 PM) mriedem: smcginnis: you're getting an endpoint not found error from ksa now
(2:26:59 PM) mriedem: smcginnis: the service type and endpoint name are 'placement' right?
(2:27:11 PM) coreycb: cdent, yep
(2:27:55 PM) mriedem: cdent: unless the ubuntu packages are going to change to take the port number out of the apache config it seems we should get the install guide updated to create the endpoint with the port number
(2:28:07 PM) cdent: mriedem: yes
(2:28:15 PM) coreycb: mriedem, cdent: yes agreed
(2:28:25 PM) cdent: that's pretty much where asettle and I landed when talking about it last week
(2:28:30 PM) mriedem: if canonical changed the apache config in a new package release, that would be confusing (at least for ocata)
(2:28:50 PM) mriedem: you could probably mirate the config somehow at install/update time for the new package, but...
(2:28:54 PM) mriedem: *migrate
(2:29:01 PM) smcginnis: Well, if our long term goal is to get rid of ports, shouldn't we make sure new services aren't getting set up that way by default?
(2:29:36 PM) mriedem: smcginnis: sure, but then canonical has to change the config in their package and provide a migration i'd think
(2:29:43 PM) mriedem: which is probably a PITA after ocata GA'ed
(2:29:58 PM) coreycb: mriedem, smcginnis: yeah we'd definitely want to do that on a release boundary if we did

Revision history for this message

Brian Moss (bmoss) wrote on 2017-04-26:

#12

Thanks guys, I really appreciate the help!

I'll change the doc back to using port numbers. In the future we can conditionalize the text if one distro changes to the service name. Of course, it makes life easier if they all decide to change at the same time. ;)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-04-26: Fix proposed to openstack-manuals (master)

#13

Fix proposed to branch: master
Review: https://review.openstack.org/459903

Changed in openstack-manuals:
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-05-04: Fix merged to openstack-manuals (master)

#14

Reviewed: https://review.openstack.org/459903
Committed: https://git.openstack.org/cgit/openstack/openstack-manuals/commit/?id=382aaf6ba65432885a9910522fd18f9685966e9d
Submitter: Jenkins
Branch: master

commit 382aaf6ba65432885a9910522fd18f9685966e9d
Author: Brian Moss <email address hidden>
Date: Wed Apr 26 10:48:38 2017 +1000

[Install] Use port numbers for Placement service

    Using the service name for the Placement endpoint
    is not yet supported in some distros. For now,
    use port 8778 instead.

Change-Id: I66ee72372bbf1d5f5fd7e69a0a8de3fe971d3bf0
Closes-Bug: #1683585

Changed in openstack-manuals:
status:	In Progress → Fix Released

Revision history for this message

Petr Kovar (pmkovar) wrote on 2017-05-04:

#15

I think this needs to be backported to Ocata, too?

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-05-04: Fix proposed to openstack-manuals (stable/ocata)

#16

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/462497

Revision history for this message

Brian Moss (bmoss) wrote on 2017-05-04: Re: [Bug 1683585] Re: Placement endpoint configuration incorrect as written

#17

On 4 May 2017 at 20:33, Petr Kovar <email address hidden> wrote:

> I think this needs to be backported to Ocata, too?
>
>
Yep, I think you're right. I'll cherry-pick it.

Thanks!

> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1683585
>
> Title:
> Placement endpoint configuration incorrect as written
>
> Status in openstack-manuals:
> Fix Released
>
> Bug description:
> Installed on vanilla ubuntu 16.04 LTS, using ocata cloud repo.
>
> - [X] This doc is inaccurate in this way: ______
>
> The guide indicates that endpoints should be created without
> specifying the port number, as so:
>
> openstack endpoint create --region RegionOne placement public
> http://controller/placement
> openstack endpoint create --region RegionOne placement admin
> http://controller/placement
> openstack endpoint create --region RegionOne placement internal
> http://controller/placement
>
> however, this does not work, and produces error messages in the nova
> logs of the compute nodes containing messages like:
>
> ERROR nova.scheduler.client.report [req-7aec8f56-4d2d-4e6f-8cb8-10af0aa06c99
> - -] Failed to create resource provider record in placement API for UUID
> 04b70101-7ec5-4078-b500-203c5560abf9. Got 404: <!DOCTYPE HTML PUBLIC
> "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>404 Not Found</title>
> </head><body>
> <h1>Not Found</h1>
> <p>The requested URL /placement/resource_providers was not found on
> this server.</p>
> <hr>
> <address>Apache/2.4.18 (Ubuntu) Server at 10.23.124.235 Port
> 80</address>
> </body></html>
>
> - [ ] This is a doc addition request.
> - [X] I have a fix to the document that I can paste below including
> example: input and output.
>
> Looking at /etc/apache2/sites-enabled/nova-placement-api.conf, I could
> clearly see that the placement api is not on port 80, but rather on
> port 8778. To fix, I went back and deleted the incorrect endpoints
> and re-created them like so:
>
> openstack endpoint create --region RegionOne placement public
> http://controller:8778
> openstack endpoint create --region RegionOne placement admin
> http://controller:8778
> openstack endpoint create --region RegionOne placement internal
> http://controller:8778
>
> After restarting the services, now the compute node could successfully
> register data with the controller and I could successfully launch
> instances.
>
> -----------------------------------
> Release: 15.0.0 on 2017-04-14 06:14
> SHA: c73899569e19fc809f56f6d58235f339c30bda2a
> Source: https://git.openstack.org/cgit/openstack/openstack-
> manuals/tree/doc/install-guide/source/nova-controller-install.rst
> URL: https://docs.openstack.org/ocata/install-guide-ubuntu/
> nova-controller-install.html
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openstack-manuals/+bug/1683585/+subscriptions
>

On 4 May 2017 at 20:33, Petr Kovar <1683585@bugs.launchpad.net> wrote:

> I think this needs to be backported to Ocata, too?
>
>
Yep, I think you're right. I'll cherry-pick it.

Thanks!

> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1683585
>
> Title:
>   Placement endpoint configuration incorrect as written
>
> Status in openstack-manuals:
>   Fix Released
>
> Bug description:
>   Installed on vanilla ubuntu 16.04 LTS, using ocata cloud repo.
>
>   - [X] This doc is inaccurate in this way: ______
>
>   The guide indicates that endpoints should be created without
>   specifying the port number, as so:
>
>   openstack endpoint create --region RegionOne placement public
> http://controller/placement
>   openstack endpoint create --region RegionOne placement admin
> http://controller/placement
>   openstack endpoint create --region RegionOne placement internal
> http://controller/placement
>
>   however, this does not work, and produces error messages in the nova
>   logs of the compute nodes containing messages like:
>
>   ERROR nova.scheduler.client.report [req-7aec8f56-4d2d-4e6f-8cb8-10af0aa06c99
> - -] Failed to create resource provider record in placement API for UUID
> 04b70101-7ec5-4078-b500-203c5560abf9. Got 404: <!DOCTYPE HTML PUBLIC
> "-//IETF//DTD HTML 2.0//EN">
>     <html><head>
>     <title>404 Not Found</title>
>     </head><body>
>     <h1>Not Found</h1>
>     <p>The requested URL /placement/resource_providers was not found on
> this server.</p>
>     <hr>
>     <address>Apache/2.4.18 (Ubuntu) Server at 10.23.124.235 Port
> 80</address>
>     </body></html>
>
>   - [ ] This is a doc addition request.
>   - [X] I have a fix to the document that I can paste below including
> example: input and output.
>
>   Looking at /etc/apache2/sites-enabled/nova-placement-api.conf, I could
>   clearly see that the placement api is not on port 80, but rather on
>   port 8778.  To fix, I went back and deleted the incorrect endpoints
>   and re-created them like so:
>
>   openstack endpoint create --region RegionOne placement public
> http://controller:8778
>   openstack endpoint create --region RegionOne placement admin
> http://controller:8778
>   openstack endpoint create --region RegionOne placement internal
> http://controller:8778
>
>   After restarting the services, now the compute node could successfully
>   register data with the controller and I could successfully launch
>   instances.
>
>   -----------------------------------
>   Release: 15.0.0 on 2017-04-14 06:14
>   SHA: c73899569e19fc809f56f6d58235f339c30bda2a
>   Source: https://git.openstack.org/cgit/openstack/openstack-
> manuals/tree/doc/install-guide/source/nova-controller-install.rst
>   URL: https://docs.openstack.org/ocata/install-guide-ubuntu/
> nova-controller-install.html
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openstack-manuals/+bug/1683585/+subscriptions
>

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-05-04: Fix merged to openstack-manuals (stable/ocata)

#18

Reviewed: https://review.openstack.org/462497
Committed: https://git.openstack.org/cgit/openstack/openstack-manuals/commit/?id=8edcbc6d79257d3c6a8396e4ebefb78632ab3d98
Submitter: Jenkins
Branch: stable/ocata

commit 8edcbc6d79257d3c6a8396e4ebefb78632ab3d98
Author: Brian Moss <email address hidden>
Date: Wed Apr 26 10:48:38 2017 +1000

[Install] Use port numbers for Placement service

    Using the service name for the Placement endpoint
    is not yet supported in some distros. For now,
    use port 8778 instead.

    Change-Id: I66ee72372bbf1d5f5fd7e69a0a8de3fe971d3bf0
    Closes-Bug: #1683585
    (cherry picked from commit 382aaf6ba65432885a9910522fd18f9685966e9d)

tags:

added: in-stable-ocata

Report a bug

This report contains Public information

Everyone can see this information.

Duplicates of this bug

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.