What would that be, you ask? My proposal is to do to the kernel what the X.Org guys did to the X server: modularize it.
So there'd be a central Linux repository that contains no hardware drivers, just core OS code, headers, build scripts, and infrastructure shared by all hardware.
Then there'd be architecture-specific repositories, so you can just check out the x86 repo or the x86-64 repo to get your desired architecture files. So never again will you have to look at source files for S/390 or Motorola 68k or OpenRISC if you just use an x86-64 CPU.
Then you'd have two repositories for each major subsystem (gpu, net, char, etc etc): one that contains drivers for hardware that is still "reasonably relevant" (as determined arbitrarily by the subsystem maintainer), and one that contains drivers for hardware that is "obsolete" (again, arbitrarily judged by the maintainer).
So if you wanted to only check out sources for a modern x86-64 desktop, the likes of which Michael reviews on Phoronix, you would have to grab several repos:
1. the base Linux repo
2. the x86-64 arch repo
3. the non-obsolete ("current") subsystem repos for the subsystems you care about, like net, gpu, or whatever else there is. If you don't use SD cards, you could completely skip that subsystem. If you don't use telephony devices, you could skip that subsystem. Defining the correct granularity of the repositories would be a practical balance: we don't want a ridiculous explosion of repositories, but we want it to be fine-grained enough that people don't get tens of megabytes of source that they'll never in a hundred years want to use.
For those who like tarballs, Linus would just tarball up each subsystem repo and release them in a directory for each RC and stable release of the kernel. So instead of a 100+ MB linux-3.0.0.tar.bz2, you'd have 20 or 30 tarballs ranging from a couple hundred kilos to a couple megs.