Anima is AirLLM's predecessor, a 33B instruction-tuned model built by fine-tuning Guanaco with QLoRA on a single H100 for 10,000 steps—a sweet-spot between training cost and model quality. The trained adapter weights and merged model are published separately on HuggingFace; training can be reproduced locally or across multiple A100/H100 GPUs via Hugging Face Accelerate. QLoRA (Quantized Low-Rank Adaptation) is a fine-tuning technique that quantizes base model weights while training only small low-rank adapter layers, drastically reducing GPU memory usage and enabling large-model fine-tuning on limited hardware. A Peft adapter stores only the fine-tuned weight differences (low-rank deltas) rather than a full model copy, enabling compact distribution of trained modifications.
The Anima model — the precursor project in this repository — is based on QLoRA fine-tuning of the 33B Guanaco model (timdettmers/guanaco-33b), trained on a single H100 GPU for 10,000 steps.[1] The choice of 10,000 steps reflects a finding from the QLoRA paper that more training samples are not always better, and that 10,000 steps offers a relatively good ROI.[1] Two HuggingFace repositories serve the Anima model: lyogavin/Anima33B (Peft adapter weights only) and lyogavin/Anima33B-merged (full merged standalone model).[1] Anima 33B training can be reproduced by installing dependencies and running ./run_Amina_training.sh, tested on a single 80 GB H100 or dual 40 GB A100 setup.[1] Multi-GPU training is supported out-of-the-box via Hugging Face Accelerate, verified on 2×A100 40 GB, where the training script runs seamlessly.[1] lyogavin/Anima33B-merged folds the Peft adapter deltas back into the base model weights, enabling dependency-free inference without needing the adapter weights separately.
Sources