在 Gitlab CI 中运行基于 selenium 的 pytest
2021-11-15
1151
我正在尝试设置一个项目，该项目应在 Gitlab CI 上运行的管道内运行用 python 编写的基于 e2e selenium 的测试。 目标是使用 pytest-docker，以便使用 docker-compose 文件启动所需的应用程序，然后我们才能运行测试（这只是为了证明我为什么使用 dind 服务和 docker/compose 映像）。 但是，我在 Gitlab CI 中运行一个简单的测试（打开 http://www.python.org 并检查标题）时遇到了问题（本地运行良好）。
所以我尝试运行的测试是这样的：
import pytest
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from webdriver_manager.firefox import GeckoDriverManager
from pytest_bdd import scenarios, given, when, then, parsers
scenarios('../features/example.feature')
@pytest.fixture
def browser():
s = Service(GeckoDriverManager().install())
options = webdriver.FirefoxOptions()
options.log.level = "TRACE"
options.add_argument('--no-sandbox')
options.add_argument('--headless')
options.add_argument('--disable-gpu')
b = webdriver.Firefox(service=s, options=options)
b.implicitly_wait(10)
yield b
b.quit()
@when('the home page is displayed')
def home_displayed(browser):
browser.get('http://www.python.org')
@then(parsers.parse('the page displays the title "{phrase}"'))
def page_displays_title(browser, phrase):
assert "Python" in browser.title
我的 gitlab-ci.yml 如下所示：
acceptance-tests:
stage: Acceptance Test
image: docker/compose
services:
- docker:dind
before_script:
- echo 'https://dl-cdn.alpinelinux.org/alpine/v3.14/community' > /etc/apk/repositories
- echo 'https://dl-cdn.alpinelinux.org/alpine/v3.14/main/' >> /etc/apk/repositories
- apk update && apk add py-pip python3-dev libffi-dev openssl-dev gcc libc-dev rust cargo make firefox-esr
- /usr/bin/python3.9 -m pip install --upgrade pip
- pip install --no-cache-dir pipenv
- pipenv install
script:
- pipenv run pytest src/backend/svfx22/tests/e2e/step_defs
在 Gitlab CI 中运行此管道步骤会导致以下堆栈跟踪：
$ pipenv run pytest src/backend/svfx22/tests/e2e/step_defs
============================= test session starts ==============================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
django: settings: svfx22.settings (from ini)
rootdir: /builds/msex20/svfx22/src/backend/svfx22, configfile: pytest.ini
plugins: bdd-4.1.0, cov-3.0.0, docker-0.10.3, django-4.4.0
collected 1 item
src/backend/svfx22/tests/e2e/step_defs/test_example.py F [100%]
=================================== FAILURES ===================================
___________________________ test_open_svf_home_page ____________________________
request = <FixtureRequest for <Function test_open_svf_home_page>>
@pytest.mark.usefixtures(*function_args)
def scenario_wrapper(request):
> _execute_scenario(feature, scenario, request)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/pytest_bdd/scenario.py:165:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/pytest_bdd/scenario.py:136: in _execute_scenario
_execute_step_function(request, scenario, step, step_func)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/pytest_bdd/scenario.py:100: in _execute_step_function
kwargs = {arg: request.getfixturevalue(arg) for arg in get_args(step_func)}
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/pytest_bdd/scenario.py:100: in <dictcomp>
kwargs = {arg: request.getfixturevalue(arg) for arg in get_args(step_func)}
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/_pytest/fixtures.py:581: in getfixturevalue
fixturedef = self._get_active_fixturedef(argname)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/_pytest/fixtures.py:601: in _get_active_fixturedef
self._compute_fixture_value(fixturedef)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/_pytest/fixtures.py:687: in _compute_fixture_value
fixturedef.execute(request=subrequest)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/_pytest/fixtures.py:1072: in execute
result = hook.pytest_fixture_setup(fixturedef=self, request=request)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/pluggy/_hooks.py:265: in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/pluggy/_manager.py:80: in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/_pytest/fixtures.py:1126: in pytest_fixture_setup
result = call_fixture_func(fixturefunc, request, kwargs)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/_pytest/fixtures.py:925: in call_fixture_func
fixture_result = next(generator)
src/backend/svfx22/tests/e2e/conftest.py:72: in browser
b = webdriver.Firefox(service=s, options=options)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/selenium/webdriver/firefox/webdriver.py:180: in __init__
RemoteWebDriver.__init__(
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:266: in __init__
self.start_session(capabilities, browser_profile)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:357: in start_session
response = self.execute(Command.NEW_SESSION, parameters)
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:418: in execute
self.error_handler.check_response(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7ffa69ad2f70>
response = {'status': 500, 'value': '{"value":{"error":"unknown error","message":"Process unexpectedly closed with status signal","stacktrace":""}}'}
def check_response(self, response: Dict[str, Any]) -> None:
"""
Checks that a JSON response from the WebDriver does not have an error.
:Args:
- response - The JSON response from the WebDriver server as a dictionary
object.
:Raises: If the response contains an error message.
"""
status = response.get('status', None)
if not status or status == ErrorCode.SUCCESS:
return
value = None
message = response.get("message", "")
screen: str = response.get("screen", "")
stacktrace = None
if isinstance(status, int):
value_json = response.get('value', None)
if value_json and isinstance(value_json, str):
import json
try:
value = json.loads(value_json)
if len(value.keys()) == 1:
value = value['value']
status = value.get('error', None)
if not status:
status = value.get("status", ErrorCode.UNKNOWN_ERROR)
message = value.get("value") or value.get("message")
if not isinstance(message, str):
value = message
message = message.get('message')
else:
message = value.get('message', None)
except ValueError:
pass
exception_class: Type[WebDriverException]
if status in ErrorCode.NO_SUCH_ELEMENT:
exception_class = NoSuchElementException
elif status in ErrorCode.NO_SUCH_FRAME:
exception_class = NoSuchFrameException
elif status in ErrorCode.NO_SUCH_WINDOW:
exception_class = NoSuchWindowException
elif status in ErrorCode.STALE_ELEMENT_REFERENCE:
exception_class = StaleElementReferenceException
elif status in ErrorCode.ELEMENT_NOT_VISIBLE:
exception_class = ElementNotVisibleException
elif status in ErrorCode.INVALID_ELEMENT_STATE:
exception_class = InvalidElementStateException
elif status in ErrorCode.INVALID_SELECTOR \
or status in ErrorCode.INVALID_XPATH_SELECTOR \
or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER:
exception_class = InvalidSelectorException
elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE:
exception_class = ElementNotSelectableException
elif status in ErrorCode.ELEMENT_NOT_INTERACTABLE:
exception_class = ElementNotInteractableException
elif status in ErrorCode.INVALID_COOKIE_DOMAIN:
exception_class = InvalidCookieDomainException
elif status in ErrorCode.UNABLE_TO_SET_COOKIE:
exception_class = UnableToSetCookieException
elif status in ErrorCode.TIMEOUT:
exception_class = TimeoutException
elif status in ErrorCode.SCRIPT_TIMEOUT:
exception_class = TimeoutException
elif status in ErrorCode.UNKNOWN_ERROR:
exception_class = WebDriverException
elif status in ErrorCode.UNEXPECTED_ALERT_OPEN:
exception_class = UnexpectedAlertPresentException
elif status in ErrorCode.NO_ALERT_OPEN:
exception_class = NoAlertPresentException
elif status in ErrorCode.IME_NOT_AVAILABLE:
exception_class = ImeNotAvailableException
elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED:
exception_class = ImeActivationFailedException
elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS:
exception_class = MoveTargetOutOfBoundsException
elif status in ErrorCode.JAVASCRIPT_ERROR:
exception_class = JavascriptException
elif status in ErrorCode.SESSION_NOT_CREATED:
exception_class = SessionNotCreatedException
elif status in ErrorCode.INVALID_ARGUMENT:
exception_class = InvalidArgumentException
elif status in ErrorCode.NO_SUCH_COOKIE:
exception_class = NoSuchCookieException
elif status in ErrorCode.UNABLE_TO_CAPTURE_SCREEN:
exception_class = ScreenshotException
elif status in ErrorCode.ELEMENT_CLICK_INTERCEPTED:
exception_class = ElementClickInterceptedException
elif status in ErrorCode.INSECURE_CERTIFICATE:
exception_class = InsecureCertificateException
elif status in ErrorCode.INVALID_COORDINATES:
exception_class = InvalidCoordinatesException
elif status in ErrorCode.INVALID_SESSION_ID:
exception_class = InvalidSessionIdException
elif status in ErrorCode.UNKNOWN_METHOD:
exception_class = UnknownMethodException
else:
exception_class = WebDriverException
if not value:
value = response['value']
if isinstance(value, str):
raise exception_class(value)
if message == "" and 'message' in value:
message = value['message']
screen = None # type: ignore[assignment]
if 'screen' in value:
screen = value['screen']
stacktrace = None
st_value = value.get('stackTrace') or value.get('stacktrace')
if st_value:
if isinstance(st_value, str):
stacktrace = st_value.split('\n')
else:
stacktrace = []
try:
for frame in st_value:
line = self._value_or_default(frame, 'lineNumber', '')
file = self._value_or_default(frame, 'fileName', '<anonymous>')
if line:
file = "%s:%s" % (file, line)
meth = self._value_or_default(frame, 'methodName', '<anonymous>')
if 'className' in frame:
meth = "%s.%s" % (frame['className'], meth)
msg = " at %s (%s)"
msg = msg % (meth, file)
stacktrace.append(msg)
except TypeError:
pass
if exception_class == UnexpectedAlertPresentException:
alert_text = None
if 'data' in value:
alert_text = value['data'].get('text')
elif 'alert' in value:
alert_text = value['alert'].get('text')
raise exception_class(message, screen, stacktrace, alert_text) # type: ignore[call-arg] # mypy is not smart enough here
> raise exception_class(message, screen, stacktrace)
E selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status signal
/root/.local/share/virtualenvs/backend-MMqkD7aq/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py:243: WebDriverException
----------------------------- Captured stderr call -----------------------------
====== WebDriver manager ======
Current firefox version is 78.15
Get LATEST geckodriver version for 78.15 firefox
There is no [linux64] geckodriver for browser in cache
Getting latest mozilla release info for v0.30.0
Trying to download new driver from https://github.com/mozilla/geckodriver/releases/download/v0.30.0/geckodriver-v0.30.0-linux64.tar.gz
Driver has been saved in cache [/root/.wdm/drivers/geckodriver/linux64/v0.30.0]
------------------------------ Captured log call -------------------------------
INFO WDM:logger.py:26
INFO WDM:logger.py:26 ====== WebDriver manager ======
INFO WDM:logger.py:26 Current firefox version is 78.15
INFO WDM:logger.py:26 Get LATEST geckodriver version for 78.15 firefox
INFO WDM:logger.py:26 There is no [linux64] geckodriver for browser in cache
INFO WDM:logger.py:26 Getting latest mozilla release info for v0.30.0
INFO WDM:logger.py:26 Trying to download new driver from https://github.com/mozilla/geckodriver/releases/download/v0.30.0/geckodriver-v0.30.0-linux64.tar.gz
INFO WDM:logger.py:26 Driver has been saved in cache [/root/.wdm/drivers/geckodriver/linux64/v0.30.0]
=========================== short test summary info ============================
FAILED src/backend/svfx22/tests/e2e/step_defs/test_example.py::test_open_svf_home_page
============================== 1 failed in 4.74s ===============================
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1
有人能帮我解决这个问题吗？我的 gitlab-ci.yml 文件中是否缺少某些内容？我尝试添加 selenium/standalone-firefox 服务，但结果相同。
2个回答
这是在 Docker 中运行 Firefox 时出现的 已知问题 。该问题已在 Firefox 84 中得到修复 。
对于 Firefox 84 之前的版本，您必须将 docker 容器的
/dev/shm
大小（默认为 64MB）增加到 Firefox 可用的大小，例如 1g 或更大。但是，据我所知，无法使用 GitLab 的共享运行器进行配置。因此，您需要为此自行托管 GitLab 运行器或为这些旧版 Firefox 使用其他 CI/浏览器提供程序。
sytech
2021-11-16
事实证明，我的问题的答案是添加 selenium/standalone-firefox 服务。我只需要在测试中正确配置我的浏览器/驱动程序。 因此，通过在我的 gitlab-ci.yml 中添加：
services:
- selenium/standalone-firefox
和：
import pytest
from selenium import webdriver
from selenium.webdriver import Remote
@pytest.fixture
def browser(http_service):
b = Remote(
command_executor='http://selenium__standalone-firefox:4444/wd/hub',
options=webdriver.FirefoxOptions()
)
b.implicitly_wait(10)
yield b
b.quit()
问题就解决了。
感谢您的回答。
mighty-mati
2021-11-17