Using selenium inside gitlab CI/CD

Question

I've desperetaly tried to set a pytest pipeline CI/CD for my personal projet hosted by gitlab.

I tried to set up a simple project with two basic files:

file test_core.py, witout any other dependencies for the sake of simplicity:

# coding: utf-8
# !/usr/bin/python3

import pytest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.options import Options

def test_basic_headless_selenium_example():
    """Test selenium installation by opening python website.
    (inspired by https://selenium-python.readthedocs.io/getting-started.html)
    """
    opts = Options()
    opts.headless = True
    driver = webdriver.Firefox(options=opts)
    driver.get("http://www.python.org")
    driver.close()

File .gitlab-ci.yml, for CI/CD automatic tests:

stages:
  - tests

pytest:python3.7:
  image: python:3.7
  stage: tests
  services:
    - selenium/standalone-firefox:latest
  script:
 #   - apt-get update && apt-get upgrade --assume-yes
    - wget -O ~/FirefoxSetup.tar.bz2 "https://download.mozilla.org/?product=firefox-latest&os=linux64"
    - tar xjf ~/FirefoxSetup.tar.bz2 -C /opt/
    - ln -s /opt/firefox/firefox /usr/lib/firefox
    - export PATH=$PATH:/opt/firefox/
    - wget -O ~/geckodriver.tar.gz "https://github.com/mozilla/geckodriver/releases/download/v0.28.0/geckodriver-v0.28.0-linux64.tar.gz"
    - tar -zxvf ~/geckodriver.tar.gz -C /opt/
    - export PATH=$PATH:/opt/
    - pip install selenium pytest
    - pytest

On my laptop, the pytestcommand works fine 100% of time. When I push a commit to gitlab, I deseperately get errors:

>       raise exception_class(message, screen, stacktrace)
E       selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 255
/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py:242: WebDriverException
=========================== short test summary info ============================
FAILED test_selenium.py::test_basic_headless_selenium_example - selenium.comm...
============================== 1 failed in 1.29s ===============================
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1

I've created a simple project: https://gitlab.com/OlivierLuG/selenium_firefox that reproduce this example. The failed pipeline can be directely found here : https://gitlab.com/OlivierLuG/selenium_firefox/-/pipelines/225711127

Does anybody have a clue how to fix this error ?

Answer 1

I've finally managed to ping gitlab CI on green with the below .gitlab-ci.yml file.

Note that I'm not a fan of yaml language. To make the file shorter, I've used a shared block of code, named install_firefox_geckodriver. Then, I've configured 2 jobs with python 3.7 and 3.8, that call this block. The keys to make this kind of test to work are: _ run in headless mode (this was already the case for me) _ install firefox and geckodriver with command lines _ install firefox dependencies _ use gitlab selenium service

Here is my yaml file. The sucessful pipeline can be found here : https://gitlab.com/OlivierLuG/selenium_firefox/-/pipelines/225756742

stages:
  - tests

.install_firefox_geckodriver: &install_firefox_geckodriver
  - apt-get update && apt-get upgrade --assume-yes
  - apt-get install gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils --assume-yes
  - wget -nv -O ~/FirefoxSetup.tar.bz2 "https://download.mozilla.org/?product=firefox-latest&os=linux64"
  - tar xjf ~/FirefoxSetup.tar.bz2 -C /opt/
  - ln -s /opt/firefox/firefox /usr/lib/firefox
  - export PATH=$PATH:/opt/firefox/
  - wget -nv -O ~/geckodriver.tar.gz "https://github.com/mozilla/geckodriver/releases/download/v0.28.0/geckodriver-v0.28.0-linux64.tar.gz"
  - tar -zxvf ~/geckodriver.tar.gz -C /opt/
  - export PATH=$PATH:/opt/


pytest:python3.7:
  image: python:3.7
  stage: tests
  services:
    - selenium/standalone-firefox:latest
  script:
    - *install_firefox_geckodriver
    - pip install selenium pytest
    - pytest

pytest:python3.8:
  image: python:3.8
  stage: tests
  services:
    - selenium/standalone-firefox:latest
  script:
    - *install_firefox_geckodriver
    - pip install selenium pytest
    - pytest