Hybrid inference, deliberately staged.
Standalone tooling for pinned llama.cpp and KTransformers/SGLang-KT profiles on AVX-512/VNNI CPUs with optional PMem-aware and NVIDIA paths.
Current state
0.0.1 established the repository. The 0.0.2
candidate publishes exact-commit target runners for human smoke; the full
dual-runtime/MoE milestone remains 0.1.0.
Model boundary
This project does not distribute or download models. Operators provision models outside the checkout; future commands may inspect an existing local path read-only.
Compute contract
AVX-512 and AVX-512 VNNI are the only universal compute gates. PMem, reserves, and NVIDIA capabilities are discovered or operator-selected.