Specify clear goals and artefacts

[qte77]

[qte77]

Entry point into project: SegFormer Quantization Part 1 Short Intro and Reason

• Why
  • Overcome constraints, e.g. TIME and SPACE
  • Examples for relevance by current models using quantization, e.g. DBRX, OlmO-Bitnet, Command R+, BLOOM, OPT
• What
  • W&B, Activations, Gradients, Conv, Embeddings
  • SegFormer Tasks (Segmentation, Classification)
• How
  • Quantization methods, e.g. PTQ, QAW, AWQ, GPTQ, GGUF/GGML, RTN
  • Precisions used, e.g. full-prec, mixed-prec, fp32, fp16, bf16, INT8, INT4, INT3, ternary, binary, INT4-g128
  • Metrics, e.g. mIoU, AP, SPACE (bytes, RAM, HDD), TIME (s), Frames (fps), Speed (ms/f)
  • Datasets, e.g. Cityscapes, ADE20K
• Discussion
  • Quantization as generalization and standardization, e.g. AWQ only Top-1%, comparable to DropOout, DropPath, Norm
  • Quantization as an inductive prior, e.g. AWQ only Top-1% as full-prec
  • Step-Function instead SoftMax, also as an Inductive Prior