On the subject of human comprehension of ten orders of magnitude:
Pretty often you have a hot path that looks like a matmul routine that does X FMAs, a physics step that takes Y matmuls, a simulation that takes Z physics steps, an optimizer that does K simulations. As a result, estimating performance across 10 orders of magnitude is just adding the logs of 4 numbers, which pretty well works out as “Count up the digits in XYZK, don’t get to 10” which is perfectly manageable to intuit
Pretty often you have a hot path that looks like a matmul routine that does X FMAs, a physics step that takes Y matmuls, a simulation that takes Z physics steps, an optimizer that does K simulations. As a result, estimating performance across 10 orders of magnitude is just adding the logs of 4 numbers, which pretty well works out as “Count up the digits in XYZK, don’t get to 10” which is perfectly manageable to intuit