Performance

The TransformersModel in src/smolagents/models.py auto-detects whether the target checkpoint is a vision-language model (VLM) by first attempting to load it with AutoModelForImageTextToText; if that raises ValueError: Unrecognized configuration class, it falls back to AutoModelForCausalLM. This means the same class handles both pure-text and multimodal local models without any user configuration.

For streaming, TransformersModel.generate_stream() spawns the model.generate() call in a background Thread and yields ChatMessageStreamDelta objects via TextIteratorStreamer. Token usage (input prompt token count + generated token count) is tracked per-delta.

VLLMModel also supports structured outputs via vLLM's StructuredOutputsParams with JSON Schema, enabling reliable structured-output generation locally. MLXModel explicitly does not support structured outputs and raises ValueError if response_format is passed.

• src/smolagents/models.py