ThousandML / Journal
Notes from Training Factory
Benchmarks, methods, and lessons from fine-tuning open models on CQ 4-bit bases.
- New8 min read
GCMP: LoRA quality with 55× fewer parameters
On Qwen3.8-27B, GCMP matched LoRA r=32 on every benchmark with 2.9M trainable parameters instead of ~159M. Plus: CQ 4-bit matches BF16 downstream, and NF4 does not.
Read the post - Guide6 min read
Train and serve your first GCMP adapter
Install Training Factory, prepare a JSONL dataset, fine-tune Qwen3.8-27B with GCMP on a CQ 4-bit base, and call it through vLLM. All on a single 48 GB GPU.
Read the post - Guide6 min read
GCMP on BF16, CQ, or NF4: choosing a base
GCMP works with or without quantization. Compare full precision, CQ 4-bit, and NF4 on quality, GPU memory, speed, and serving support.
Read the post - Guide5 min read
One base, many adapters: serving GCMP at scale
Serve several ~11 MB GCMP adapters from one CQ base in vLLM, pick one per request, and batch requests for different adapters together.
Read the post - Corrected7 min read
QLoRA on CQ vs. Unsloth NF4: an earlier loss comparison
Our first CQ vs. NF4 fine-tuning run. The training-log losses in this post are not on the same scale, so we no longer use them. The downstream benchmarks in the GCMP post replace them.
Read the post - Fine-tuning8 min read
Fine-tune on a quantized base. Just make sure the base is good.
Why training adapters on a 4-bit base lowers cost enough to fine-tune in-house, and why a calibration-free CQ base is a strong place to start.
Read the post