What is a quantized model?
If you have ever seen AI models before installation, you have probably come across something like: “mistral-7b-instruct-v0.3.Q4_K_M.gguf”. The model name may ring a bell, we have already explained the meaning of the letter “b”, but what is that string of letters and numbers at the end?… That is quantization. A simple but clever technique that…
