Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I don't really buy that they're even necessarily useful for comparing models. In the example from the article, if model A says "48 + 6 minutes" and gets marked correct, and model B says "63 minutes" (the correct answer) and gets marked correct, the test will say that they're equivalent on that axis when in fact one gave a completely nonsense answer.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: