Test fixtures for Garak are configured via pytest plugins (pytest-mock, respx, pytest-asyncio, pytest-cov, pytest_httpserver, requests-mock) and global setup in conftest.py that suppresses noise, initializes the plugin cache, and provides reusable fixtures like mitigation_outputs and loaded_intent_service. Custom pytest markers (requires_storage, integration) and autouse fixtures (config_cleanup) ensure test isolation and skip tests when storage resources are unavailable.
The tests optional dependency group in pyproject.toml includes pytest>=9.1, pytest-mock>=3.14.0, respx>=0.21.1, pytest-asyncio>=0.21.0, pytest-cov>=5.0.0, pytest_httpserver>=1.1.0, and requests-mock==1.12.1.[1] pytest is configured in pyproject.toml to suppress all warnings by default and re-enable only warnings originating from the garak namespace.[1]
The tests/conftest.py global setup redirects garak logging to /dev/null (via os.devnull) unless the GARAK_LOG_FILE environment variable is set, preventing spurious log output during test runs.[2] tests/conftest.py ensures the garak plugin cache file exists at test-suite startup by calling _plugins.PluginCache.instance() if the user cache file is missing.[2]
The config_cleanup autouse fixture in tests/conftest.py runs for every test and, via LIFO finalizers, reloads _config, clears PluginProvider._instance_cache, and removes report, hitlog, and HTML files produced during the test.[2] The mitigation_outputs fixture in tests/conftest.py returns a (COMPLYING_OUTPUTS, REFUSAL_OUTPUTS) tuple — two lists of representative LLM outputs — for use in detector and probe tests.[2] The loaded_intent_service fixture in tests/conftest.py loads the garak config and then calls garak.services.intentservice.load(), providing a ready IntentService for tests that need it — see Context Aware Scanning for the feature this supports.[2]
tests/conftest.py registers two custom pytest markers: requires_storage and integration.[2] The requires_storage marker defaults to requiring 1 GB free on the root filesystem (/); tests decorated with it are skipped with a clear message if that threshold is not met.[2]
Sources