"what is 16929481231+22312333222?" is an easy way to test this claim. Pick large enough numbers and there's no way all the sums of that size would fit into the dataset (you don't need to stick to + either, but it's the simplest thing that works)
For my contribution to the conversation: Earlier/cheaper models can't do it either, they make mistakes, they need a calculator/jupiter kernel/what have you. 'Medium' models will put the numbers underneath each other and do it 'properly' in a table, checking themselves after. Claude Opus 4.6 (the current rolls royce today) just says the answer in one go sometimes (it's a monster). But all of them end up spending many seconds and thousands of tokens on a task that takes a calculator or an ALU fractions of a second.
