geckodriver version is incompatible with firefox in ppc64el

[Jesse Simpson]

Hi all,

An application my team and I are developing makes use of a firefox selenium driver in python for ppc64le architectures. I noticed that this module in our application was not functioning due to an inability for the python library to connect to the geckodriver. I also noticed that the x86_64 version of our app is functioning with the same version of firefox and geckodriver.

The python error we are seeing is the following:

TimeoutException: Message: connection refused

The timeout was set to 10 seconds, so I updated it to 40, and then 300 and got the same issue.

geckodriver starts up without any errors. but selenium can't seem to connect to it.

Upon deploying an old version of a docker container of our app (based on ubuntu 18.04), I compared versions of the old version, which selenium worked, vs the new version which selenium does not work. What I found was that although the apt packages for firefox and geckodrivers seem to point to a v75.0 build of firefox, the version in geckodriver did not change between version upgrades.

so,

firefox version changed 72.0.2 -> 75.0
geckodriver did not 36.0 -> 36.0

I suspect this is the source of the issue, because if firefox changes and the geckodriver remains the same, they won't sync up right.

========= Versions on non-working image (version from Apr 30, 2020 on ppc64le) =================

root@0d1a13cbda24:/usr/src/# lsb_release -rd
Description: Ubuntu 18.04.3 LTS
Release: 18.04

root@0d1a13cbda24:/usr/src/# apt-cache policy firefox
firefox:
  Installed: 75.0+build3-0ubuntu0.18.04.1
  Candidate: 75.0+build3-0ubuntu0.18.04.1
  Version table:
 *** 75.0+build3-0ubuntu0.18.04.1 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main ppc64el Packages
        500 http://ports.ubuntu.com/ubuntu-ports bionic-security/main ppc64el Packages
        100 /var/lib/dpkg/status
     59.0.2+build1-0ubuntu1 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main ppc64el Packages

root@0d1a13cbda24:/usr/src/# apt-cache policy firefox-geckodriver
firefox-geckodriver:
  Installed: 75.0+build3-0ubuntu0.18.04.1
  Candidate: 75.0+build3-0ubuntu0.18.04.1
  Version table:
 *** 75.0+build3-0ubuntu0.18.04.1 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main ppc64el Packages
        500 http://ports.ubuntu.com/ubuntu-ports bionic-security/main ppc64el Packages

root@0d1a13cbda24:/usr/src/# firefox --version
Mozilla Firefox 75.0

root@0d1a13cbda24:/usr/src/# geckodriver --version
geckodriver 0.26.0

root@0d1a13cbda24:/usr/src/# pip show selenium

if you can create the driver successfully, you've made it past our bug.
Name: selenium
Version: 3.141.0
Summary: Python bindings for Selenium
Home-page: https://github.com/SeleniumHQ/selenium/
Author: UNKNOWN
Author-email: UNKNOWN
License: Apache 2.0
Location: /usr/local/lib/python3.7/dist-packages
Requires: urllib3
Required-by:

======= Versions on working image (image from jan 29th, 2020 on ppc64le) ==================

root@b5d05f80b923:/usr/src/# lsb_release -rd
Description: Ubuntu 18.04.3 LTS
Release: 18.04

root@b5d05f80b923:/usr/src/# apt-cache policy firefox
firefox:
  Installed: 72.0.2+build1-0ubuntu0.18.04.1
  Candidate: 72.0.2+build1-0ubuntu0.18.04.1
  Version table:
 *** 72.0.2+build1-0ubuntu0.18.04.1 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main ppc64el Packages
        500 http://ports.ubuntu.com/ubuntu-ports bionic-security/main ppc64el Packages
        100 /var/lib/dpkg/status
     59.0.2+build1-0ubuntu1 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main ppc64el Packages

root@b5d05f80b923:/usr/src/# apt-cache policy firefox-geckodriver
firefox-geckodriver:
  Installed: 72.0.2+build1-0ubuntu0.18.04.1
  Candidate: 72.0.2+build1-0ubuntu0.18.04.1
  Version table:

if you can create the driver successfully, you've made it past our bug.
 *** 72.0.2+build1-0ubuntu0.18.04.1 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main ppc64el Packages
        500 http://ports.ubuntu.com/ubuntu-ports bionic-security/main ppc64el Packages
        100 /var/lib/dpkg/status

root@b5d05f80b923:/usr/src/# firefox --version
Mozilla Firefox 72.0.2

root@b5d05f80b923:/usr/src/# geckodriver --version
geckodriver 0.26.0

root@b5d05f80b923:/usr/src/# pip show selenium
Name: selenium
Version: 3.141.0
Summary: Python bindings for Selenium
Home-page: https://github.com/SeleniumHQ/selenium/
Author: UNKNOWN
Author-email: UNKNOWN
License: Apache 2.0
Location: /usr/local/lib/python3.7/dist-packages
Requires: urllib3
Required-by:

===== Test python code to try =======

# docker run -it -u 0 --rm ubuntu:18.04 bash

then run the following:

    > apt update -y && apt upgrade -y
    > apt install python3-venv python3.7 python3.7-venv firefox firefox-geckodriver
    > python3.7 -m venv env
    > source env/bin/activate
    > pip install selenium==3.141.0
    > python3.7

paste in the code below:

from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException, TimeoutException
from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

SELENIUM_TIMEOUT = 40

def _get_profile():
    profile = FirefoxProfile()
    return profile

def _get_driver():
    options = FirefoxOptions()
    options.headless = True
    profile = _get_profile()
    _driver = webdriver.Firefox(options=options, firefox_profile=profile, executable_path='geckodriver')
    _driver.set_page_load_timeout(SELENIUM_TIMEOUT)

driver = _get_driver()

if you can create the driver successfully, you've made it past our bug. Otherwise, it fails on

>>> driver = _get_driver()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in _get_driver
  File "/env/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/env/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/env/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: connection refused

[Olivier Tilloy (osomon)]

I can reproduce the problem in a ppc64el cloud instance, and it can be observed in firefox's autopkgtest results, too (http://autopkgtest.ubuntu.com/packages/firefox/focal/ppc64el).

The problem is not a version mismatch though. The firefox-geckodriver package is built from the same source package as firefox, and it works on other architectures.