Over the years I have often thought the same thing. I work in the Civil Engineering field and have spent lots of time on job sites seeing scrapers, blades, compactors, dozers, backhoes, and other equipment running around. But I have the feeling it's not as easy as it looks.
I have worked with lots of backhoe operators over the years and most are pretty decent. I did run into one that it took over half a day to do the job (test pits) when an experienced operator could have finished it in less than an hour, easy. If you don't know what you are doing, yeah, you can probably move some scoops around randomly, etc, but actually doing anything intentional would probably take a lot of time.
Have I lied to you? I mean, in this room? Trust me, leave that thing alone. - GLaDOS
Did IQs just drop sharply while I was away? - Ripley