I don't buy it. If that's true, how come all of the top results I get are slow, heavy bloated sites, yet small, simple, fast sites are ~10 pages deep?

Sure, speed should be a ranking factor, but I'd much rather have a slow site that's relevant than a fast one that isn't.

How about a relevant page that is fast. We can have the best of both worlds.

Wouldn't that be nice. But Google don't control how relevant or fast something is, they can only decide how to rank them.

... thats my point.

This is a problem that Google created. If they would prioritize small, simple, fast sites, and rank them higher, people would make more sites like that. Google is the one pushing slow, bloated sites to the top of the list. Now they are coming in to "fix" the problem that they made.