Quantizationgiáo trình tự học
Chương 152 phút đọc

15Tài liệu tham khảo#

Nền tảng

  1. M. Horowitz. Computing's energy problem (and what we can do about it). ISSCC 2014.
  2. B. Jacob et al. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. CVPR 2018. arXiv:1712.05877.
  3. R. Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv:1806.08342, 2018.
  4. M. Nagel et al. A White Paper on Neural Network Quantization. arXiv:2106.08295, 2021.
  5. A. Gholami et al. A Survey of Quantization Methods for Efficient Neural Network Inference. arXiv:2103.13630, 2021.
  6. S. Migacz. 8-bit Inference with TensorRT. NVIDIA GTC 2017.

Huấn luyện và PTQ nâng cao

  1. Y. Bengio, N. Léonard, A. Courville. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv:1308.3432, 2013.
  2. S. K. Esser et al. Learned Step Size Quantization. ICLR 2020. arXiv:1902.08153.
  3. M. Nagel et al. Data-Free Quantization Through Weight Equalization and Bias Correction. ICCV 2019. arXiv:1906.04721.
  4. M. Nagel et al. Up or Down? Adaptive Rounding for Post-Training Quantization. ICML 2020. arXiv:2004.10568.

LLM và định dạng số mới

  1. T. Dettmers et al. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. NeurIPS 2022. arXiv:2208.07339.
  2. G. Xiao et al. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. ICML 2023. arXiv:2211.10438.
  3. E. Frantar et al. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. ICLR 2023. arXiv:2210.17323.
  4. J. Lin et al. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. MLSys 2024. arXiv:2306.00978.
  5. T. Dettmers et al. QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS 2023. arXiv:2305.14314.
  6. Z. Liu et al. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache. ICML 2024. arXiv:2402.02750.
  7. P. Micikevicius et al. FP8 Formats for Deep Learning. arXiv:2209.05433, 2022.
  8. B. D. Rouhani et al. Microscaling Data Formats for Deep Learning. arXiv:2310.10537, 2023.

Tài liệu công cụ

  1. PyTorch, Quantization (docs.pytorch.org) và torchao (github.com/pytorch/ao).
  2. TensorFlow / LiteRT, Post-training quantization (ai.google.dev/edge/litert).