Garak supports multiple generator backends beyond its built-in models: REST endpoints via RestGenerator, Hugging Face and Cohere via their APIs, and local ggml models, each configured with environment variables or command flags. Multimodal support for vision tasks and image-based jailbreaks like FigStep enables testing of multimodal LLM vulnerabilities. ggml is a tensor library for running quantized LLM weights locally on CPU, without requiring a GPU or network connectivity.
The rest.RestGenerator can connect to any REST endpoint returning plaintext or JSON and is configured via a short YAML file describing the endpoint; full reference docs are at https://reference.garak.ai/en/latest/garak.generators.rest.html.[1] The Hugging Face generator optionally reads the HF_INFERENCE_TOKEN environment variable for API authentication.[1] The Cohere generator (--target_type cohere) defaults to the command model when --target_name is omitted, and reads the COHERE_API_KEY environment variable for authentication.[1] The ggml generator (--target_type ggml) reads the GGML_MAIN_PATH environment variable for the path to the ggml main executable.[1] Multimodal (vision/image) support — including LLaVA and the FigStep jailbreak — was introduced in v0.9.0.13.[2] The ggml generator (--target_type ggml) runs fully offline, performing local inference without API calls or network connectivity.
Sources