network-config-service.sh incorrect for Fedora Atomic 25

[Andrew Bruce]

While deploying new bay to OpenStack Mitaka, found that some generated files were incorrect:

./magnum/drivers/common/templates/kubernetes/fragments/network-config-service.sh
./magnum/drivers/common/templates/kubernetes/fragments/write-network-config.sh
./magnum/drivers/common/templates/swarm/fragments/network-config-service.sh

The problem is the usage of FLANNEL_ETCD / FLANNEL_ETCD_KEY after sourcing in /etc/sysconfig/flanneld. For Fedora Atomic 25, these variables are completely different.

Upon reviewing the latest code in the OpenStack Magnum repo, I see that both the Kubernetes fragments appear updated. However, the Swarm fragment still uses:

<cut>
if ! [ "$FLANNEL_ETCD" ] && [ "$FLANNEL_ETCD_KEY" ]; then
    echo "ERROR: missing required configuration" >&2
    exit 1
fi

echo "creating flanneld config in etcd"
while ! curl -sf -L $ETCD_CURL_OPTIONS \
    $FLANNEL_ETCD/v2/keys${FLANNEL_ETCD_KEY}/config \
    -X PUT --data-urlencode value@${FLANNEL_JSON}; do
    echo "waiting for etcd"
    sleep 1
done
</cut>

These need to be changed to FLANNEL_ETCD_ENDPOINTS / FLANNEL_ETCD_PREFIX as was done with Kubernetes.

While in there, I request three additional changes that still affect the Swarm / Kubernetes network-config-service.sh fragments:

1. Please source in current settings from /etc/sysconfig/flanneld

2. In the fragments, escape variable references. For example, how about:

<cut>
if ! [ "\$FLANNEL_ETCD_ENDPOINTS" ] && [ "\$FLANNEL_ETCD_PREFIX" ]; then
    echo "ERROR: missing required configuration" >&2
    exit 1
fi

echo "creating flanneld config in etcd"
while ! curl -sf -L \$FLANNEL_ETCD_ENDPOINTS/v2/keys\${FLANNEL_ETCD_PREFIX}/config \
  -X PUT --data-urlencode value@${FLANNEL_JSON}; do
    echo "waiting for etcd"
    sleep 1
done
</cut>

3. For both the Kubernetes and the Swarm network-config-service.sh, can you put in a dependency when setting up /etc/systemd/system/flannel-config.service? Something like:

<cut>
cat > $FLANNEL_CONFIG_SERVICE <<EOF
[Unit]
After=etcd.service
Requires=etcd.service
# ABr: maintain state
Before=flanneld.service

[Service]
Type=oneshot
EnvironmentFile=/etc/sysconfig/flanneld
ExecStart=$FLANNEL_CONFIG_BIN

[Install]
WantedBy=multi-user.target
EOF
</cut>

That last setting will prevent flanneld from starting too early - settings will be properly populated to etcd!

Thanks very much.

[Spyros Trigazis (strigazi)]

Mitaka is not updated for the latest fedora atomic.

Mitaka supports an old image. We need to update. I recommend Newton or later.

[Spyros Trigazis (strigazi)]

The FLANNEL variable name you propose are fixed in:
newton https://review.openstack.org/#/c/464078/

and

ocata or later https://review.openstack.org/#/c/417457/

[Spyros Trigazis (strigazi)]

Please NOTE: this bug affects only mitaka.

[Andrew Bruce (andybrucenet)]

I believe this means I should remove this bug. We are working on K8s-hosted OpenStack with Ocata so I can verify this fix further on in my testing.

[Andrew Bruce (andybrucenet)]

I will verify this fix on Ocata as part of a stackanetes deployment.

[Spyros Trigazis (strigazi)]

Thanks!

At the moment we try to have Newton up to date with the latest images. Do you need mitaka to support the latest fedora as well?

[Andrew Bruce (andybrucenet)]

No I do not - I'm working on integrating Kolla-Kubernetes which is at Ocata so I can verify the process with Ocata.

I have my hacks / workarounds for Mitaka.

Thanks,
Andy

> On May 25, 2017, at 9:44 AM, Spyros Trigazis <email address hidden> wrote:
>
> Thanks!
>
> At the moment we try to have Newton up to date with the latest images.
> Do you need mitaka to support the latest fedora as well?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1679301
>
> Title:
> network-config-service.sh incorrect for Fedora Atomic 25
>
> Status in Magnum:
> New
> Status in Magnum mitaka series:
> In Progress
>
> Bug description:
> While deploying new bay to OpenStack Mitaka, found that some generated
> files were incorrect:
>
> ./magnum/drivers/common/templates/kubernetes/fragments/network-config-service.sh
> ./magnum/drivers/common/templates/kubernetes/fragments/write-network-config.sh
> ./magnum/drivers/common/templates/swarm/fragments/network-config-service.sh
>
> The problem is the usage of FLANNEL_ETCD / FLANNEL_ETCD_KEY after
> sourcing in /etc/sysconfig/flanneld. For Fedora Atomic 25, these
> variables are completely different.
>
> Upon reviewing the latest code in the OpenStack Magnum repo, I see
> that both the Kubernetes fragments appear updated. However, the Swarm
> fragment still uses:
>
> <cut>
> if ! [ "$FLANNEL_ETCD" ] && [ "$FLANNEL_ETCD_KEY" ]; then
> echo "ERROR: missing required configuration" >&2
> exit 1
> fi
>
> echo "creating flanneld config in etcd"
> while ! curl -sf -L $ETCD_CURL_OPTIONS \
> $FLANNEL_ETCD/v2/keys${FLANNEL_ETCD_KEY}/config \
> -X PUT --data-urlencode value@${FLANNEL_JSON}; do
> echo "waiting for etcd"
> sleep 1
> done
> </cut>
>
> These need to be changed to FLANNEL_ETCD_ENDPOINTS /
> FLANNEL_ETCD_PREFIX as was done with Kubernetes.
>
> While in there, I request three additional changes that still affect
> the Swarm / Kubernetes network-config-service.sh fragments:
>
> 1. Please source in current settings from /etc/sysconfig/flanneld
>
> 2. In the fragments, escape variable references. For example, how
> about:
>
> <cut>
> if ! [ "\$FLANNEL_ETCD_ENDPOINTS" ] && [ "\$FLANNEL_ETCD_PREFIX" ]; then
> echo "ERROR: missing required configuration" >&2
> exit 1
> fi
>
> echo "creating flanneld config in etcd"
> while ! curl -sf -L \$FLANNEL_ETCD_ENDPOINTS/v2/keys\${FLANNEL_ETCD_PREFIX}/config \
> -X PUT --data-urlencode value@${FLANNEL_JSON}; do
> echo "waiting for etcd"
> sleep 1
> done
> </cut>
>
> 3. For both the Kubernetes and the Swarm network-config-service.sh,
> can you put in a dependency when setting up /etc/systemd/system
> /flannel-config.service? Something like:
>
> <cut>
> cat > $FLANNEL_CONFIG_SERVICE <<EOF
> [Unit]
> After=etcd.service
> Requires=etcd.service
> # ABr: maintain state
> Before=flanneld.service
>
> [Service]
> Type=oneshot
> EnvironmentFile=/etc/sysconfig/flanneld
> ExecStart=$FLANNEL_CONFIG_BIN
>
> [Install]
> WantedBy=multi-user.target
> EOF
> </cut>
>
> That last setting will prevent flanneld...

[Javier Castillo (javcasalc)]

Hi, this bugs still afects swarm clusters (Ocata and Master trunks)

for example:

 - https://github.com/openstack/magnum/blob/master/magnum/drivers/common/templates/swarm/fragments/network-config-service.sh#L50

 - https://github.com/openstack/magnum/blob/master/magnum/drivers/common/templates/swarm/fragments/network-config-service.sh#L57

I think this should be something similar to:

if [ -z "$FLANNEL_ETCD_ENDPOINTS" ] || [ -z "$FLANNEL_ETCD_PREFIX" ]; then
  echo "ERROR: missing required configuration" >&2
  exit 1
fi
echo "creating flanneld config in etcd"
while ! curl -sf -L $ETCD_CURL_OPTIONS \
        $FLANNEL_ETCD_ENDPOINTS/v2/keys${FLANNEL_ETCD_PREFIX}/config \
        -X PUT --data-urlencode value@${FLANNEL_JSON}; do
    echo "waiting for etcd"
    sleep 1
done

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/507822

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/507822
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=010a2673d4424358199878f717dcec6159276701
Submitter: Zuul
Branch: master

commit 010a2673d4424358199878f717dcec6159276701
Author: Javier Castillo Alcíbar <email address hidden>
Date: Wed Sep 27 13:10:42 2017 +0200

    Swarm: Incorrect reference to Flannel variables

    Swarm network-config-service makes reference to
    wrong variables:
    - FLANNEL_ETCD instead of FLANNEL_ETCD_ENDPOINTS
    - FLANNEL_ETCD_KEY instead of FLANNEL_ETCD_PREFIX

    Change-Id: Ie9f1913177f4a5fd803bcf8de1de66384395ecdd
    Closes-Bug: #1679301

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/magnum 6.0.0 release.