Non-python tests with a command-line option in pytest

Question

I'm implementing custom test cases that are based on external files using the tutorial from https://docs.pytest.org/en/latest/example/nonpython.html.

I need to parametrise them with one bool flag. I'd like to be able to run pytest with a commandline option, in my case --use-real-api, which would turn using mocks off and do the real talking to a remote network API.

I've tried using the cmdopt tutorial and blend them together, but can't find any way to read the parameter from within the custom pytest.Item subclass. Could you help? Here is a trivial example from the tutorial. I'd like to get it to change the test behaviour depending on the value of the cmdopt passed.

# content of conftest.py
import pytest


def pytest_collect_file(parent, path):
    if path.ext == ".yml" and path.basename.startswith("test"):
        return YamlFile(path, parent)


class YamlFile(pytest.File):
    def collect(self):
        import yaml
        raw = yaml.safe_load(self.fspath.open())
        for name, spec in sorted(raw.items()):
            yield YamlItem(name, self, spec)


class YamlItem(pytest.Item):
    def __init__(self, name, parent, spec):
        super().__init__(name, parent)
        self.spec = spec

    def runtest(self):
        for name, value in sorted(self.spec.items()):
            # some custom test execution (dumb example follows)
            if name != value:
                raise YamlException(self, name, value)

    def repr_failure(self, excinfo):
        """ called when self.runtest() raises an exception. """
        if isinstance(excinfo.value, YamlException):
            return "\n".join(
                [
                    "usecase execution failed",
                    "   spec failed: %r: %r" % excinfo.value.args[1:3],
                    "   no further details known at this point.",
                ]
            )

    def reportinfo(self):
        return self.fspath, 0, "usecase: %s" % self.name


class YamlException(Exception):
    """ custom exception for error reporting. """


def pytest_addoption(parser):
    parser.addoption(
        "--cmdopt", action="store", default="type1", help="my option: type1 or type2"
    )


@pytest.fixture
def cmdopt(request):
    return request.config.getoption("--cmdopt")

Answer 1

Each collection entity in pytest (File, Module, Function etc) is a subtype of the Node class which defines access to the config object. Knowing that, the task becomes easy:

def pytest_addoption(parser):
    parser.addoption('--run-yml', action='store_true')

def pytest_collect_file(parent, path):
    run_yml = parent.config.getoption('--run-yml')
    if run_yml and path.ext == ".yml" and path.basename.startswith("test"):
        return YamlFile(path, parent)

Running pytest --run-yml will now collect the YAML files; without the flag, they are ignored.

Same for accessing the config in custom classes, for example:

class YamlItem(pytest.Item):    
    def runtest(self):
        run_yml = self.config.getoption('--run-yml')
        ...

etc.