Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Hey, I'm the post author. This is a totally fair point! I do think though that depending on your specific requirements open-source models can be a 10x+ improvement. For example, we serve Mistral 7B for less than 1/10th the cost of GPT-4-Turbo, which is the model most of our users are comparing us to.


I serve ~300tk/s of Mistral 7B for $0.60/hr by renting a cloud 3090. That's a lot cheaper than GPT-4-Turbo, though the quality is closer to GPT-3.5.

Mixtral 8x7b is closer to GPT-4 quality though and only 2x the compute requirement of Mistral 7B.


This is the 10x I was looking for. Great post by the way!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: