The pxd harness exhaustively runs all specified probes against all specified detectors in a cross product, skipping individual load failures to complete the run rather than abort. A buff in Garak is a post-processing plugin that transforms or augments probe outputs before they are evaluated by detectors.
The pxd harness in garak/harnesses/pxd.py runs all specified probes against all specified detectors — a full cross product, sorted alphabetically — and warns that this thoroughness may produce detector–probe pairings that are semantically mismatched, because not all detectors are designed to pick up failure modes in all situations.[1] PxD.run() accepts an optional buff_names parameter (defaulting to []) and loads buffs via self._load_buffs(buff_names) before iterating over the sorted probe and detector names.[1]
During probe loading, PxD.run() skips a probe with a logged error if _plugins.load_plugin() raises an exception, and skips it with a logged warning if load_plugin returns a falsy value — allowing the rest of the run to continue in both cases.[1] Detector loading uses break_on_fail=False; any detector that fails to load is logged as an error and skipped rather than aborting the run.[1]
For each successfully loaded probe, PxD.run() delegates execution to super().run() — the base Harness — passing the full detector list and announce_probe=False, processing one probe at a time (see Harness base for base-class behavior).[1]
Sources