AirLLMQWen and AirLLMQWen2 are thin AirLLMBaseModel subclasses that implement AirLLM's streaming inference protocol for the Qwen family by mapping layer names, managing rotary embeddings, and packing KV caches in Qwen's layer_past convention.
AirLLMQWen (in air_llm/airllm/airllm_qwen.py) and AirLLMQWen2 (in air_llm/airllm/airllm_qwen2.py) are both thin subclasses of AirLLMBaseModel — see AirLLMBaseModel — that disable BetterTransformer/optimum by returning False from get_use_better_transformer().[1][2] AirLLMQwen3_5 (in air_llm/airllm/airllm_qwen3_5.py) is a thin subclass of AirLLMBaseModel that implements AirLLM's streaming inference protocol for the Qwen3.8-27B dense vision-language model — the first Qwen3.x VL variant in the codebase. The Qwen3.8-27B model (served via AirLLMQwen3_5) runs on an RTX 3090 GPU consuming 3.33 GB of GPU memory through AirLLM's layer-streaming inference. AirLLMQwen4Exp (in air_llm/airllm/airllm_qwen4_exp.py) is a thin subclass of AirLLMBaseModel that implements AirLLM's streaming inference protocol for the Qwen3.8-Flash-Next model. The Qwen3.8-Flash-Next model (served via AirLLMQwen4Exp) runs on consumer hardware consuming as little as 5.95 GB of GPU memory through AirLLM's layer-streaming inference.
AirLLMQWen maps its layer names to QWen's transformer block naming scheme via set_layer_names_dict: embed → transformer.wte, layer_prefix → transformer.h, norm → transformer.ln_f, lm_head → lm_head.[2]
AirLLMQWen.get_generation_config returns a bare GenerationConfig() with no special settings.[2]
AirLLMQWen.get_pos_emb_args computes rotary_emb._ntk_alpha_cached_list on the model's transformer and returns a rotary_pos_emb_list argument dict used when passing positional embeddings to each QWen layer during streaming.[2] AirLLMQWen.get_past_key_value_args packs the KV cache as {'layer_past': (k_cache, v_cache)}, matching QWen's layer_past parameter convention (distinct from ChatGLM's kv_cache convention, covered on ChatGLM).[2] AirLLMQWen.get_past_key_values_cache_seq_len reads the cached sequence length from axis 1 of the key tensor (past_key_values[0][0].shape[1]), unlike ChatGLM which reads from axis 0.[2]
AirLLMQWen.get_attention_mask_args always passes attention_mask=None to QWen layers, and get_position_ids_args returns an empty dict — QWen's attention and position handling is driven entirely by rotary embeddings, not explicit masks or position IDs.[2]
test_qwen3_8_split.py serves as a template for validating on-disk splitting behaviour for large models. airllm_qwen4_exp.py and test_qwen38_flash_next_split.py together serve as a reference pattern for extending AirLLM to new Qwen-lineage models via the layer-streaming and on-disk-split pipeline.
Sources