How does GPTQ work?

The technique applies layer-by-layer quantization with calibration data, optimizing weights to minimize the squared error of layer outputs rather than just the weights themselves. GPTQ produces quantized models that are typically within 1-2 perplexity points of FP16 baselines on most language modeling benchmarks, despite using only 25-30% of the storage. The platform strengthens enterprise re