Selection resolution converts a run.spec string into concrete probe and buff names via resolve_spec in garak/_selection.py, applying tier and tag filters with AND logic, then excludes, and validating intent codes. The resolver returns a Resolution dataclass with selected probes and buffs, rejected selectors, inactive items, and intent metadata; probes without explicit tiers default to tier 9, and excludes always override includes regardless of order.
garak/_selection.py is the single module that resolves a run.spec Spec object into concrete probe and buff names; the grammar and parsing live in garak/_spec.py and the plugin registry lives in garak/_plugins.py.[1] resolve_spec in garak/_selection.py is the single entry point used by the CLI and harnesses to turn a Spec into a Resolution containing selected probe names, buff names, rejected selectors, and intent codes.[1] resolve_spec returns a _spec.Resolution dataclass with fields: selected (a dict holding sorted probes and buffs lists), rejected, inactive, empty_reason, intents, blocked_intents, and intents_explicit.[1] A Spec is the parsed representation of a run.spec string, produced by garak/_spec.py; resolve_spec in garak/_selection.py accepts a Spec as its primary input, making _spec.py parsing a prerequisite for constructing selection requests.
Probes that do not declare a tier are assigned _DEFAULT_TIER = 9 (equivalent to Tier.UNLISTED) by garak/_selection.py.[1] Tier and tag positive filters are combined with AND logic: a probe must satisfy all specified filters to remain a candidate.[1] Probes can also be selected by MISP tag (e.g. owasp:llm01), a feature introduced in v0.9.0.10.[2] Excludes are applied last, and exclude always wins over include regardless of order.[1] A tier exclude selector removes probes of exactly that tier number — not a ceiling — and logs a debug message if no probes of that tier exist.[1]
Intent specifiers * and all (case-insensitive) are accepted without validation as vacuous sentinels that IntentService expands to every intent; all other codes are validated via _spec.validate_intent_specifier, and invalid codes are appended to rejected.[1]
By default resolve_spec raises ValueError listing all unknown selectors; passing skip_unknown=True suppresses the error and continues with the resolved subset.[1]
Sources