AirLLMMistral and AirLLMMixtral are minimal subclasses of AirLLMBaseModel that disable BetterTransformer and use default generation configs, deferring all other behavior to the base class. Both models avoid the optimum-based BetterTransformer in favor of Transformers' built-in sdpa acceleration. SDPA (Scaled Dot-Product Attention) is a fused attention kernel built into PyTorch that delivers acceleration comparable to BetterTransformer without requiring the external optimum library.
AirLLMMistral (in air_llm/airllm/airllm_mistral.py) is a minimal subclass of AirLLMBaseModel that only disables BetterTransformer and returns a bare GenerationConfig(); all other behavior is inherited from the base class — see AirLLMBaseModel for the shared interface.[1] AirLLMMixtral (in air_llm/airllm/airllm_mixtral.py) explicitly disables BetterTransformer by returning False from get_use_better_transformer(), reflecting the drop of the optimum-based BetterTransformer dependency in favour of built-in sdpa.[2] AirLLMMixtral returns a bare GenerationConfig() (all defaults) from get_generation_config(), relying on Transformers' own generation defaults without any Mixtral-specific overrides.[2]
Sources