bitnet
Official inference framework for 1-bit LLMs
BitNet is the official inference framework for 1-bit LLMs (BitNet b1.58). It provides optimized kernels for fast and lossless inference of 1.58-bit quantized models on CPU, with architecture-specific optimizations for x86 (TL2) and ARM (TL1) platforms.
homepage ↗ github: microsoft/BitNet
Available in
| Overlay | Newest | Ebuilds | Last activity | |
|---|---|---|---|---|
| bennypowers GitHub ↗ | 9999 | 1 | 5 d | details › |
Versions & arches
Use flags of 9999
- +openmp Build support for the OpenMP (support parallel computing), requires >=sys-devel/gcc-4.2 built with USE="openmp"
2 expansion flags (python targets, ABIs, cpu flags…)
- cpu_flags_x86_avx2
- cpu_flags_arm_neon