Glossary
Quantisation
A technique that reduces the precision of a model’s numbers so it takes up less memory and runs on more modest computers, at the cost of some quality.
Example
A 4-bit version of a model that fits on a laptop.
Common mistake
Always picking the most compressed version so it “fits”. Below 4 bits quality usually drops sharply.