bitsandbytes
Subscription: Developer
Active
Subscription: Developer
A library providing k-bit optimizers and matrix multiplication routines for efficient quantization of large language models, enabling 8-bit and 4-bit inference and training on consumer GPUs with significantly reduced memory requirements.