FM
github.com
Flash-MoE: 397B on a Laptop
Running a big model on a small laptop.
About Flash-MoE: 397B on a Laptop
Flash-MoE runs a 397B Mixture-of-Experts model on a laptop with a pure C/Metal inference engine. Streams a 209GB model from SSD with 4-bit quantization and no Python for production-quality output.
Categories
Topics
Published
Discussion highlights
AI-generated summary based on Hacker News comments
🔗 Full discussion: https://news.ycombinator.com/item?id=47476422
No comments yet. Have you tried it?