Generator test contracts live in tests/generators/test_generators.py and enforce structural invariants (every DEFAULT_PARAMS key in _supported_params), signature matching for generate() and _call_model(), and correct list-length/type behavior on parallel calls. Several generator classes are excluded from general instantiation tests due to environmental or model-name constraints: AzureOpenAIGenerator, BedrockGenerator, WatsonXGenerator, function-based and file-based variants, and third-party-dependent generators.
Generator structural and behavioural contracts live in tests/generators/test_generators.py. Several classes are excluded from the general instantiation tests because they require special environments: AzureOpenAIGenerator (extra env vars), BedrockGenerator (Bedrock-specific model names), WatsonXGenerator (extra env vars), function.Multiple/function.Single (mock local functions), GgmlGenerator (file validation on disk), NeMoGuardrails (third-party install dependency), HuggingFace variants, and LangChainLLMGenerator (model name restrictions).[1]
The structural test in tests/generators/test_generators.py enforces that every key present in a generator's DEFAULT_PARAMS also appears in its _supported_params.[1] For non-OpenAI/Groq/Azure/NeMoGuardrailsServer generators, the same test file enforces that generate() is annotated as (prompt: Conversation) -> List[Union[None, Message]] and that _call_model() carries matching signature annotations.[1]
test_parallel_requests verifies that generators.test.Lipsum returns exactly the requested number of Message objects with non-empty text when generations_this_call=3 is passed to generate().[1]
Example: canonical generator parallel call test using generators.test.Lipsum in tests/generators/test_generators.py:
prompt = Conversation(Turn("user", [Message("this is a test")]))
result = g.generate(prompt=prompt, generations_this_call=3)
assert isinstance(result, list)
assert len(result) == 3
assert all(isinstance(item, Message) for item in result)
When generator output consists entirely of skip-sequence content (nothing outside the delimiters), generators.test.Repeat returns an empty Message(''); truncated (unclosed) skip sequences are also stripped — both behaviours are asserted in tests/generators/test_generators.py.[1]
Sources