TLS params not set for session

Bug #1820083 reported by Dan Streetman
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-etcd3gw (Ubuntu)
Fix Released
Medium
Unassigned
Bionic
Fix Released
Medium
Heather Lemon
Cosmic
Won't Fix
Medium
Unassigned
Disco
Won't Fix
Medium
Unassigned
Eoan
Won't Fix
Medium
Unassigned
Focal
Fix Released
Medium
Heather Lemon
Groovy
Won't Fix
Medium
Unassigned
Hirsute
Fix Released
Medium
Unassigned

Bug Description

[Impact]

A connection session is opened, but the TLS parameters (timeout, ca, cert and key) are not actually set for the session. This prevents use of TLS for the etcd3gw package.

[Test Plan]

# Create self signed certs, using the default for all prompts

$ openssl req -addext "subjectAltName = DNS:localhost" -x509 -keyout localhost.key -newkey rsa:4096 -nodes -sha256 -out localhost.crt

# install 'etcd' package, stop the default server, and spin up ectd server

$ sudo apt install etcd
$ sudo systemctl stop etcd

$ etcd --name test --data-dir test --cert-file=localhost.crt --key-file=localhost.key --advertise-client-urls=https://localhost:2379 --listen-client-urls=https://localhost:2379

# run test script

$ cat test.py
#!/usr/bin/python3

from etcd3gw import Etcd3Client

c = Etcd3Client(host="localhost", protocol="https", cert_key="localhost.key", cert_cert="localhost.crt", ca_cert="localhost.crt", timeout=10)
c.put('test', 'success!')
resp = c.get('test')
print(b''.join(resp).decode())

$ ./test.py
success!

[Where Problems Could Occur]

This adds TLS parameters (if provided) to the session, so regressions would involve failed connections, possibly those without TLS that had TLS params incorrectly provided before.

[Other]

the upstream bug is https://github.com/dims/etcd3-gateway/issues/20
fixed upstream with pull request https://github.com/dims/etcd3-gateway/pull/21
via commit 90b7a19cdc4daa1230d7f15c10b113abdefdc8c0

that commit is contained in version 0.2.2 which is already in h, so this is needed in b/f/g. This package was not included in Xenial.

Dan Streetman (ddstreet)
Changed in python-etcd3gw (Ubuntu Bionic):
importance: Undecided → Medium
Changed in python-etcd3gw (Ubuntu Cosmic):
importance: Undecided → Medium
Changed in python-etcd3gw (Ubuntu Disco):
importance: Undecided → Medium
status: New → In Progress
Changed in python-etcd3gw (Ubuntu Cosmic):
status: New → In Progress
Changed in python-etcd3gw (Ubuntu Bionic):
status: New → In Progress
Changed in python-etcd3gw (Ubuntu Disco):
assignee: nobody → Dan Streetman (ddstreet)
Changed in python-etcd3gw (Ubuntu Bionic):
assignee: nobody → Dan Streetman (ddstreet)
Changed in python-etcd3gw (Ubuntu Cosmic):
assignee: nobody → Dan Streetman (ddstreet)
Dan Streetman (ddstreet)
description: updated
Changed in python-etcd3gw (Ubuntu Cosmic):
status: In Progress → Won't Fix
Changed in python-etcd3gw (Ubuntu Disco):
status: In Progress → Won't Fix
Changed in python-etcd3gw (Ubuntu Focal):
status: In Progress → New
Changed in python-etcd3gw (Ubuntu Bionic):
status: In Progress → New
assignee: Dan Streetman (ddstreet) → nobody
Changed in python-etcd3gw (Ubuntu Cosmic):
assignee: Dan Streetman (ddstreet) → nobody
Changed in python-etcd3gw (Ubuntu Disco):
assignee: Dan Streetman (ddstreet) → nobody
Changed in python-etcd3gw (Ubuntu Focal):
assignee: Dan Streetman (ddstreet) → nobody
Changed in python-etcd3gw (Ubuntu Eoan):
importance: Undecided → Medium
tags: added: sts
Dan Streetman (ddstreet)
tags: added: sts-sponsor-volunteer
Revision history for this message
Brian Murray (brian-murray) wrote :

The Eoan Ermine has reached end of life, so this bug will not be fixed for that release

Changed in python-etcd3gw (Ubuntu Eoan):
status: New → Won't Fix
Changed in python-etcd3gw (Ubuntu Bionic):
assignee: nobody → Heather Lemon (hypothetical-lemon)
Changed in python-etcd3gw (Ubuntu Focal):
assignee: nobody → Heather Lemon (hypothetical-lemon)
Changed in python-etcd3gw (Ubuntu Bionic):
status: New → In Progress
Changed in python-etcd3gw (Ubuntu Focal):
status: New → In Progress
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "lp1820083-tlsparams-bionic.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
description: updated
description: updated
Mathew Hodson (mhodson)
Changed in python-etcd3gw (Ubuntu Groovy):
importance: Undecided → Medium
Revision history for this message
Mathew Hodson (mhodson) wrote :

Fixed in Ubuntu Hirsute.
---

python-etcd3gw (0.2.5-1) unstable; urgency=medium

  [ Ondřej Nový ]
  * Run wrap-and-sort -bastk.

  [ Thomas Goirand ]
  * Switch to new repo URL.
  * New upstream release (Closes: #980004).
  * Removed 0001_reproducible-build.patch applied upstream.
  * Add python3-mock as build-depends.

 -- Thomas Goirand <email address hidden> Wed, 13 Jan 2021 09:49:04 +0100

Changed in python-etcd3gw (Ubuntu Hirsute):
status: New → Fix Released
description: updated
description: updated
description: updated
description: updated
description: updated
Changed in python-etcd3gw (Ubuntu Groovy):
status: New → Triaged
status: Triaged → In Progress
assignee: nobody → Heather Lemon (hypothetical-lemon)
Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

debdiff for groovy

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

groovy debdiff

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

bionic debdiff

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

focal debdiff

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

Resubmitted patches for verification

Dan Streetman (ddstreet)
tags: added: sts-sponsor-ddstreet
removed: sts-sponsor-volunteer
Dan Streetman (ddstreet)
tags: added: sts-sponsor-slashd
removed: sts-sponsor-ddstreet
Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

revert test case description

description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :
Download full text (6.4 KiB)

# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import os
import ssl
import socket

import threading

from OpenSSL import crypto
from etcd3gw.client import Etcd3Client
from etcd3gw.tests import base
from future.backports.http.server import (HTTPServer as _HTTPServer,
                                          SimpleHTTPRequestHandler, BaseHTTPRequestHandler)

class ETCDMock(_HTTPServer):

    def __init__(self, server_address, handler_class, context):
        _HTTPServer.__init__(self, server_address, handler_class)
        self.context = context

    def __str__(self):
        return ('<%s %s:%s>' %
                (self.__class__.__name__,
                 self.server_name,
                 self.server_port))

    def get_request(self):
        try:
            sock, addr = self.socket.accept()
            sslconn = self.context.wrap_socket(sock, server_side=True)
            self.sock = sock
        except socket.error as e:
            print("failure in etcdservermock: %s" % e)
            exit(1)
        return sslconn, addr

class ETCDMockRequestHandler(SimpleHTTPRequestHandler):
    protocol_version = "HTTP/1.0"

    def do_GET(self):
        if self.path == "/health":
            example_response = b"{health:true}"
            self.send_response(200)
            self.send_header("Content-Type", "application/json")
            self.send_header("Content-Length", len(example_response))
            self.end_headers()
            self.wfile.write(example_response)
        else:
            super().do_GET()

    def do_POST(self):
        if self.path == "/maintenance/status":
            example_response = b"{health:true}"
            self.send_response(200)
            self.send_header("Content-Type", "application/json")
            self.send_header("Content-Length", len(example_response))
            self.end_headers()
            self.wfile.write(example_response)
        else:
            super().do_POST()

class ETCDServerThread(threading.Thread):

    def __init__(self, context):

        self.flag = None
        self.server = ETCDMock(('127.0.0.1', 2379),
                               ETCDMockRequestHandler,
                               context)
        self.port = self.server.server_port
        threading.Thread.__init__(self)
        self.daemon = True

    def __str__(self):
        return "<%s %s>" % (self.__class__.__name__, self.server)

    def start(self, flag=None):
        self.flag = flag
        threading.Thread.start(self)

    def run(self):
        if self.flag:
            self.flag.set()
        try:
            self.server.serve_forever(0.05)
        finally:
            self.server.server_close()

    def stop(self):
    ...

Read more...

description: updated
description: updated
Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

updated unit test code

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

attached format-patch unit test changes

description: updated
description: updated
Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

deleted all previous attachements

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

updated focal debdiff patch (03-02-2021)

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

debdiff for groovy (03-03-2021)

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

debdiff bionic (03-02-2021)

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

redo groovy control file is alphabetized (03-02-2021)

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

redo focal debdiff alphabetized control file

description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
tags: added: sts-sponser
Dan Streetman (ddstreet)
description: updated
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote :

The Groovy Gorilla has reached end of life, so this bug will not be fixed for that release

Changed in python-etcd3gw (Ubuntu Groovy):
status: In Progress → Won't Fix
Dan Streetman (ddstreet)
tags: added: sts-sponsor
removed: sts-sponser
Revision history for this message
Eric Desrochers (slashd) wrote :

[sts-sponsor]

The debdiff add a new build-depends for python-openssl[0].

It seems like you made a patch of yours[1] (UBUNTU SAUCE ?? I can't find in the upstream project[2]) since your patch requires crypto in OpenSSL module[3].

Could you elaborate and provide rationale for this patch ? And why this is needed here ?
Ideally, I would prefer not having build-depends in stable release.

- Eric

[0] "+ python3-openssl,"

[1]
From 4db59e0620c3696ad654145e33a0ea5e6529b817 Mon Sep 17 00:00:00 2001
From: Heather Lemon <email address hidden>
Date: Thu, 25 Feb 2021 16:50:40 -0700
Subject: create new unit test for https etcd server

[2] https://opendev.org/openstack/etcd3gw/commit/4db59e0620c3696ad654145e33a0ea5e6529b817

[3] - ++from OpenSSL import crypto

Changed in python-etcd3gw (Ubuntu Groovy):
assignee: Heather Lemon (hypothetical-lemon) → nobody
Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

Hi Eric,

I had added a unit test for better code coverage and a requirement of another LP#1900617 whose unittests were failing, but that might not be needed now. I am still investigating this, Currently launchpad is down so I will be working on this tomorrow as well.

Thanks,
Heather Lemon

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

Revised patch for tls-params removed added unit testing coverage for certs - focal

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

revised debdiff without added unit tests, bionic

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

adds attachment patch file Set-transport-options-on-requests

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

pastebin of proposed code changes

https://pastebin.canonical.com/p/45jWbygmSV/

Revision history for this message
Eric Desrochers (slashd) wrote :

[sts-sponsors]

It is looking definitely better.

Here's some nitpicking:

# For both Focal and Bionic:

- From: =?UTF-8?q?Tade=C3=A1=C5=A1=20Urs=C3=ADny?= <email address hidden>
+ from: Tadeas Ursíny <email address hidden>

- Origin: upstream, https://github.com/dims/etcd3-gateway/commit/90b7a19cdc4daa1230d7f15c10b113abdefdc8c0
+ Origin: upstream, https://opendev.org/openstack/etcd3gw/commit/90b7a19

-Bug-Ubuntu: https://bugs.launchpad.net/+bug/1820083
+ Bug-Ubuntu: https://bugs.launchpad.net/bugs/1820083

In order to follow the patch convention name already in place in the src package:
# quilt rename -P lp1820083-Set-transport-options-on-requests-session.patch 0002-Set-transport-options-on-requests-session.patch

(Don't forget to adjust d/changelog accordingly)

# For Bionic is already used in Eoan:
0.2.1-1ubuntu1

Version
https://launchpad.net/ubuntu/+source/python-etcd3gw/0.2.1-1ubuntu1

Please modify the version for not conflicting with another release having the same version already (even if EOL'd).

0.2.1-1ubuntu0.18.04.1 might be preferable here.

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :
Download full text (3.5 KiB)

For the record this is the proposed unit test to be added. Since the pastebin is set to expire after one year.

# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

import os
from socket import gethostname

# from OpenSSL import crypto
from etcd3gw.client import Etcd3Client
from etcd3gw.tests import base

def create_self_signed_cert():
    # create a key pair
    pub_key = crypto.PKey()
    pub_key.generate_key(crypto.TYPE_RSA, 2048)

    # create a csr
    csr = crypto.X509Req()
    csr.get_subject().C = "US"
    csr.get_subject().ST = "Boston"
    csr.get_subject().L = "Boston"
    csr.get_subject().O = "Test Company Ltd"
    csr.get_subject().OU = "Test Company Ltd"
    csr.get_subject().CN = gethostname()
    csr.set_pubkey(pub_key)
    csr.sign(pub_key, "sha256")

    # create a self-signed cert
    cert = crypto.X509()
    cert.get_subject().C = "US"
    cert.get_subject().ST = "Boston"
    cert.get_subject().L = "Boston"
    cert.get_subject().O = "Test Company Ltd"
    cert.get_subject().OU = "Test Company Ltd"
    cert.get_subject().CN = gethostname()
    cert.set_serial_number(1000)
    cert.gmtime_adj_notBefore(0)
    cert.gmtime_adj_notAfter(10 * 365 * 24 * 60 * 60)
    cert.set_issuer(cert.get_subject())
    cert.set_pubkey(pub_key)
    cert.sign(pub_key, "sha256")

    with open('cert.crt', 'w') as crt:
        if crt is not None:
            crt.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert).decode("utf-8"))
    with open('test.key', 'w') as key:
        if key is not None:
            key.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, pub_key).decode("utf-8"))
    with open('test.ca', 'w') as ca:
        if ca is not None:
            ca.write(crypto.dump_certificate_request(crypto.FILETYPE_PEM, csr).decode("utf-8"))

    crt.close()
    key.close()
    ca.close()

class TestEtcd3Gateway(base.TestCase):

    def test_client_default(self):
        client = Etcd3Client()
        self.assertEqual("http://localhost:2379/v3alpha/lease/grant",
                         client.get_url("/lease/grant"))

    def test_client_ipv4(self):
        client = Etcd3Client(host="127.0.0.1")
        self.assertEqual("http://127.0.0.1:2379/v3alpha/lease/grant",
                         client.get_url("/lease/grant"))

    def test_client_ipv6(self):
        client = Etcd3Client(host="::1")
        self.assertEqual("http://[::1]:2379/v3alpha/lease/grant",
                         client.get_url("/lease/grant"))

    def test_client_tls(self):
        create_self_signed_cert()
        with open('cert.crt', 'r') as crt_file, \
                open('test.key', 'r') as key_file, \
                open('test.ca', 'r') as ca_file:
            client = Etc...

Read more...

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

Revised bionic patch addressing comments the only change I didn't apply was changing the github to opendev source. Can I ask why we're making this change? Thanks

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

Revised focal patch addressing comments

Revision history for this message
Eric Desrochers (slashd) wrote :

Because:

1) We want to make sure we give credit to the author by making sure his/her name is readable.
2) The Bug-Ubuntu URL you originally added redirected to a 404 ERROR.
3) The dims' github repo last update was back in 2020, so it seems like a mirror or no-longer maintained source location.
4) The version was already picked for Eoan, in order to avoid conflict, I prefer not having 2 identical versions for more than 1 release (Even if Eoan is EOL).

Hope it answers your questions.

Revision history for this message
Eric Desrochers (slashd) wrote :

5) The patch rename is to continue the logic in the patch convention name already existing in the src code.

Revision history for this message
Eric Desrochers (slashd) wrote :

I see that dims is the maintainer[0], so the github repo should be fine and be considered as a trusted source.

We can leave it as is.

[0] - https://pypi.org/project/etcd3gw/

Revision history for this message
Eric Desrochers (slashd) wrote :

[sts-sponsors]

Uploaded in Focal and Bionic upload queues.
It is now waiting for the SRU verification team to approve the src package to start building and become available in the -proposed pockets for the testing/verification phase.

Thanks for your contribution Heather.

- Eric

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

Ah I should have been more explict, my only question was around renaming of the github repo to opendev.

Also, there is another LP that's supposed to go with this one: https://bugs.launchpad.net/ubuntu/focal/+source/python-etcd3gw/+bug/1900617

Revision history for this message
Eric Desrochers (slashd) wrote :

[sts-sponsors]

The upload mentioned in comment #38 has been rejected as per my request.

Re-uploaded in Focal and Bionic upload queues, including both (LP: #1820083) & (LP: #1900617)

It is now waiting for the SRU verification team to approve the src package to start building and become available in the -proposed pockets for the testing/verification phase.

Thanks for your contribution, Heather.

- Eric & Dariusz

Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Dan, or anyone else affected,

Accepted python-etcd3gw into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-etcd3gw/0.2.1-3ubuntu1.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in python-etcd3gw (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
Changed in python-etcd3gw (Ubuntu Bionic):
status: In Progress → Fix Committed
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Dan, or anyone else affected,

Accepted python-etcd3gw into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-etcd3gw/0.2.1-1ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed-bionic
Revision history for this message
Heather Lemon (hypothetical-lemon) wrote (last edit ):

#testing steps for python-etcd3gw focal
python version - Python 3.9.7
version tested - python-etcd3gw 0.2.1-3ubuntu1.20.04.1

pull-lp-source python-etcd3gw focal

# generate certs in python-etcd3gw folder
openssl req -x509 -keyout localhost.key -newkey rsa:4096 -nodes -sha256 -out localhost.crt

#download etcd binaries & launch etcd locally with TLS enabled
wget https://github.com/etcd-io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz

tar -zxvf etcd-v3.3.14-linux-amd64.tar.gz

cd etcd-v3.3.14-linux-amd64/

#spin up etcd server
./etcd --name infra0 --data-dir infra0 --cert-file=/python-etcd3gw/localhost.crt --key-file=/python-etcd3gw/localhost.key --advertise-client-urls=https://127.0.0.1:2379 --listen-client-urls=https://127.0.0.1:2379

# test cert connection endpoint
curl --cacert localhost.crt --key localhost.key --cert localhost.crt https://127.0.0.1:2379/health

response
{"health":"true"}

modify test.py to your config setup for etcd

#run test
./test.py

response
success!

# code for test.py
touch test.py
chmod +rwx test.py

gedit test.py
#!/usr/bin/python3

from etcd3gw import Etcd3Client

c = Etcd3Client(host="127.0.0.1", protocol="https", cert_key="localhost.key", cert_cert="localhost.crt", ca_cert="localhost.crt", timeout=10)
c.put('test', 'success!')
resp = c.get('test')
print(b''.join(resp).decode())

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

I am getting a PBR error when testing this with bionic

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

Commenting out the __init__.py inside line 25/26.
where it checks the version with PBR
#__version__ = pbr.version.VersionInfo(
# 'etcd3gw').version_string()

commenting this line out and running ./test.py
my response is success.

# testing steps for bionic
version - 0.2.1-1ubuntu0.18.04.1
python3 version - 3.6.9
python2 version - 2.7.17

mkdir 1820083-verification-testing
cd 1820083-verification-testing
pull-lp-source python-etcd3gw bionic

apt install python3-pip
pip3 install -r requirements.txt
pip3 install -r test-requirements.txt

# generate certs in python-etcd3gw folder
openssl req -x509 -keyout localhost.key -newkey rsa:4096 -nodes -sha256 -out localhost.crt

#download etcd binaries & launch etcd locally with TLS enabled
wget https://github.com/etcd-io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz

tar -zxvf etcd-v3.3.14-linux-amd64.tar.gz

cd etcd-v3.3.14-linux-amd64/

#spin up etcd server
./etcd --name infra0 --data-dir infra0 --cert-file=/python-etcd3gw/localhost.crt --key-file=/python-etcd3gw/localhost.key --advertise-client-urls=https://127.0.0.1:2379 --listen-client-urls=https://127.0.0.1:2379

# test cert connection endpoint
curl --cacert localhost.crt --key localhost.key --cert localhost.crt https://127.0.0.1:2379/health

response
{"health":"true"}

modify test.py to your config setup for etcd

#run test
./test.py

response
success!

# code for test.py
touch test.py
chmod +rwx test.py

gedit test.py
#!/usr/bin/python3

from etcd3gw import Etcd3Client

c = Etcd3Client(host="127.0.0.1", protocol="https", cert_key="localhost.key", cert_cert="localhost.crt", ca_cert="localhost.crt", timeout=10)
c.put('test', 'success!')
resp = c.get('test')
print(b''.join(resp).decode())

#also feel free to run unit tests with
python3 -m unittest

tags: added: verification-done-bionic
removed: verification-needed-bionic
Revision history for this message
Eric Desrochers (slashd) wrote :

Is there a way you can verify the actual binary package ? Instead of testing it by pulling the source code ?

Verification need to be done on the binary package, this is what we will promote in -updates.

- Eric

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

TEST CASE:
1. Use Bionic series lxc container
2. Enable proposed repo in /etc/apt/sources.list
3. deb http://archive.ubuntu.com/ubuntu bionic-proposed main universe
4. sudo apt-get update
5. apt-get install python-etcd3gw=0.2.1-1ubuntu0.18.04.1
6. openssl req -x509 -keyout localhost.key -newkey rsa:4096 -nodes -sha256 -out localhost.crt
7. ./etcd --name infra0 --data-dir infra0 --cert-file=/root/python-etcd3gw-0.2.1/localhost.crt --key-file=/root/python-etcd3gw-0.2.1/localhost.key --advertise-client-urls=https://127.0.0.1:2379 --listen-client-urls=https://127.0.0.1:2379
8. apt-get install etcd-client
9. etcdctl --endpoints https://127.0.0.1:2379 --ca-file=localhost.crt --cert-file=localhost.crt --key-file=localhost.key member list
10. response - 8e9e05c52164694d: name=infra0 peerURLs=http://localhost:2380 clientURLs=https://127.0.0.1:2379 isLeader=true
11. extra testing - attempt to connect without certs gives
Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:4001: connect: connection refused
; error #1: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02"
12. etcdctl --endpoints https://127.0.0.1:2379 --ca-file=localhost.crt --cert-file=localhost.crt --key-file=localhost.key set foo bar
13. etcdctl --endpoints https://127.0.0.1:2379 --ca-file=localhost.crt --cert-file=localhost.crt --key-file=localhost.key get foo
response = bar

VERIFICATION DONE
you can connect to the etcd server with certificate infomation provided
8e9e05c52164694d: name=infra0 peerURLs=http://localhost:2380 clientURLs=https://127.0.0.1:2379 isLeader=true

Troubleshooting

1. if you have this error 140647060033984:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/root/.rnd
# comment out the RANDFILE at the top of /etc/ssl/openssl.cnf
RANDFILE = $ENV::HOME/.rnd

2. If you have this error client: etcd cluster is unavailable or misconfigured; error #0: x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs
# under /etc/ssl/openssl.cnf add your IP for your cert
[ v3_ca ]
subjectAltName = IP:127.0.0.1

# also note, you can update ca-certificates
cp localhost.crt /usr/share/ca-certificates/
sudo update-ca-certificates
sudo dpkg-reconfigure ca-certificates

3. Try rebooting openssl service after making changes or container

4. make sure etcd is running
Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: connect: connection refused
error #0: dial tcp 127.0.0.1:2379: connect: connection refused

Revision history for this message
Eric Desrochers (slashd) wrote :

Thanks, Heather.

Don't forget to do the verification for the other bug as well (LP: #1900617)

- Eric

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

TEST CASE:
1. use focal series lxc container
2. Enable proposed repo in /etc/apt/sources.list
3. deb http://archive.ubuntu.com/ubuntu focal-proposed main universe
4. sudo apt-get update
5. apt install etcd-client
6. apt-get install python3-etcd3gw #Focal has renamed this package with python3-etcd3gw
7. wget https://github.com/etcd-io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz
8. tar -xvf etcd-v3.3.13-linux-amd64.tar.gz
9. openssl req -x509 -keyout localhost.key -newkey rsa:4096 -nodes -sha256 -out localhost.crt
10. ./etcd --name infra0 --data-dir infra0 --cert-file=/root/python-etcd3gw-0.2.1/localhost.crt --key-file=/root/python-etcd3gw-0.2.1/localhost.key --advertise-client-urls=https://127.0.0.1:2379 --listen-client-urls=https://127.0.0.1:2379
11. curl --cacert localhost.crt --key localhost.key --cert localhost.crt https://127.0.0.1:2379/health
    response - {"health":"true"}
12. etcdctl --endpoints https://127.0.0.1:2379 --ca-file=localhost.crt --cert-file=localhost.crt --key-file=localhost.key member list
    response = 8e9e05c52164694d: name=infra0 peerURLs=http://localhost:2380 clientURLs=https://127.0.0.1:2379 isLeader=true
13. etcdctl --endpoints https://127.0.0.1:2379 --ca-file=localhost.crt --cert-file=localhost.crt --key-file=localhost.key set foo bar
 response - bar

VERIFICATION DONE
you can connect to the etcd server with certificate information provided

TROUBLESHOOTING
** See above troubleshooting steps

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-etcd3gw - 0.2.1-3ubuntu1.20.04.1

---------------
python-etcd3gw (0.2.1-3ubuntu1.20.04.1) focal; urgency=medium

  * d/p/lp1820083-Set-transport-options-on-requests-session.patch
    - Sets TLS parameters for session (LP: #1820083)
  * d/p/0001-lp1900617-When-gateway-sends-failure-response-include-text-in-.patch
    - Include response text in raised exception
    d/p/0002-lp1900617-Include-resp.text-as-detail-in-all-etcd-exceptions.patch
    - Add new unit test for return exception
    d/p/0003-lp1900617-Fix-exception-signature.patch
    - Derived exceptions can use arguments again
    (LP: #1900617)

 -- Heather Lemon <email address hidden> Mon, 07 Dec 2020 12:21:25 -0700

Changed in python-etcd3gw (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for python-etcd3gw has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-etcd3gw - 0.2.1-1ubuntu0.18.04.1

---------------
python-etcd3gw (0.2.1-1ubuntu0.18.04.1) bionic; urgency=medium

  * d/p/lp1820083-set-transport-options-on-requests-session.patch
    - Sets TLS parameters for session (LP: #1820083)

  * d/p/0001-lp1900617-When-gateway-sends-failure-response-include-text-in.patch
    - Include response text in raised exception
    d/p/0002-lp1900617-Include-resp.text-as-detail-in-all-etcd-exceptions.patch
    - Add new unit test for return exception
    d/p/0003-lp1900617-Fix-exception-signature.patch
    - Derived exceptions can use arguments again
    (LP: #1900617)

 -- Heather Lemon <email address hidden> Wed, 27 Oct 2021 15:59:44 +0000

Changed in python-etcd3gw (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.