AirLLM streams model layers from disk through VRAM rather than keeping the full model resident, enabling low VRAM usage — dispatch through AutoModel.get_module_class automatically selects the appropriate subclass based on model architecture. On non-macOS platforms, core entry points (AirLLMBaseModel, AutoModel, split_and_save_layers, NotEnoughSpaceException) are exported from air_llm/airllm/__init__.py; on macOS, only AirLLMLlamaMlx and AutoModel are available, with AutoModel.from_pretrained always returning the MLX backend.
AirLLM achieves low VRAM usage by streaming model layers — and, for MoE models, individual experts — from disk through VRAM rather than keeping the full model resident.[1]
On non-macOS platforms, air_llm/airllm/__init__.py exports AirLLMBaseModel, AutoModel, split_and_save_layers, and NotEnoughSpaceException as the core entry points.[2] On macOS (platform == 'darwin'), air_llm/airllm/__init__.py exports only AirLLMLlamaMlx and AutoModel; no PyTorch-based subclasses are imported.[2]
AutoModel in air_llm/airllm/auto_model.py is a dispatch-only factory: it cannot be instantiated directly and raises EnvironmentError if you try.[3] AutoModel.get_module_class is the entry point for dispatch: it accepts a model repo ID or local path and returns a (module, class_name) tuple for the appropriate AirLLM subclass.[4] To load gated models, pass an hf_token kwarg; AutoModel.get_module_class forwards it as the token= argument to AutoConfig.from_pretrained.[3] AutoModel.from_pretrained on macOS always returns an AirLLMLlamaMlx instance, bypassing the architecture-dispatch table entirely — see macOS MLX backend for details.[3]
AutoModel.get_module_class reads the model's config.architectures[0] field to select a class, defaulting to AirLLMBaseModel for any architecture not found in ARCH_OVERRIDES.[3] ARCH_OVERRIDES in auto_model.py maps six architecture strings to dedicated subclasses — ChatGLMModel and ChatGLMForConditionalGeneration → AirLLMChatGLM; QWenLMHeadModel → AirLLMQWen; BaichuanForCausalLM and BaiChuanForCausalLM → AirLLMBaichuan; InternLMForCausalLM → AirLLMInternLM; KimiK3ForConditionalGeneration → AirLLMKimiK3 — and every other standard *ForCausalLM falls through to AirLLMBaseModel without any code change.[3] The dispatch table tested in air_llm/tests/test_automodel.py maps representative repo IDs to concrete class names: garage-bAInd/Platypus2-7B → AirLLMLlama2, Qwen/Qwen-7B → AirLLMQWen, internlm/internlm-chat-7b → AirLLMInternLM, THUDM/chatglm3-6b-base → AirLLMChatGLM, baichuan-inc/Baichuan2-7B-Base → AirLLMBaichuan, mistralai/Mistral-7B-Instruct-v0.1 → AirLLMMistral, mistralai/Mixtral-8x7B-v0.1 → AirLLMMixtral.[4] AirLLMLlama2 in air_llm/airllm/airllm.py is a trivial subclass of AirLLMBaseModel with no overrides; it exists as a named entry point for backward compatibility.[5]
Sources