I have read nearly everything Dan has written in the past 3+ years, and that has to be his most confusing/poor written article ever when it comes to understanding of people who are tech smart, just do not know much info on that specific subject (like say, mem controllers)
Hrm, I guess I know enough about OS's, device drivers, memory models, etc then -- it wasn't confusing at all to me. And I didn't know the exact details about why various bits of memory between 3 and 4 GB were being reserved; but I do now. (I knew it was for memory mapped I/O, but not the specifics).
Dan says that only certain hardware can access that upper level of memory, like say a video card. Then, would it not make sense in some form with your 121729712 MB of videocard memory to use that space, and not take up the "free mem" available to standard basic applications?
I'm not sure I follow what you're asking.
There are two (slightly) different concepts occurring here -- first, memory mapped I/O. Basically device drivers allow programs to access their memory by saying "use location X" -- when you access location X then the call is intercepted by the device driver and you read/write to its own memory instead. This isn't just for video cards (although in a typical system they have the largest chunk of MMIO), but also for your hard drives, optical drives, network card, USB, etc.
The second concept, for Windows only, is that the OS reserves a large portion of the address space for itself. On a standard XP/Vista system, addresses from 0-2GB are available to programs, from 2GB-3GB is for Windows itself (system libraries and such), and 3GB-4GB is MMIO. As Dan notes, you can tell Windows to let applications use up to 3GB, in which case Windows has to try to stuff everything into the 3-4GB range (which, as he notes, may not be a wise course of action).
In any case the video card is never using up space available to "standard basic applications" in Windows. At least not unless you have >1 GB of video card memory (and other MMIO).
If you're asking "well, ok, so why did they put the MMIO crap in the 3-4GB range anyway?" -- well, if not there, then where? It has to be somewhere between 0 and 4 GB for 32-bit applications; they cannot address outside of that (yes, even with PAE's magic tricks -- PAE simply maps 4GB chunks at a time into the 32-bit addressable range; at it does it at a rather steep cost even then, and requires special programming). Given that, putting the MMIO at the end of the 4GB range and mapping downward is sane. Much the same was done with the 8086 and it's 1 MB of address space (thus, the 640K limit).
You could ditch MMIO entirely, of course. If you don't mind your system running about as fast as a 486 with PIO drives -- because that's what you're talking about doing. MMIO allows, amongst other things, DMA capabilities -- so that the CPU doesn't have to be involved in every single byte being moved from a device to main memory or vica versa. Write a block of memory to your video card's MMIO and the CPU will stop caring once the device driver has figured out what you're doing (a few hundred instructions at most); then the video card will talk directly to main memory and copy all the data itself (which is potentially millions of instructions; or billions once you start talking about 1GB+ of video memory). There's a reason why we've kept the MMIO concept (in its various forms) around for decades -- it's really, really freaking fast and the only way to do it sanely.
I could go search Wiki and the internet for it, or read some of my many books...but ehh, I don't wanna...you do it!
No! At no point in writing this did I actually need to do any research. That's how much of a nerd I am. So there.