Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

For decode, MoE is nice for either bs=1 (decoding for a single user), or bs=<very large> (do EP to efficiently serve a large amount of users).

Anything in between suffers.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: