I'm guessing both of you went with non-combat oriented characters? The leveling system is a real bitch for that type - it assumes that you've got better at combat, but if you are leveling up because of passive skills, you won't have.
Nah, the skills that level up the most for me are Restoration, Destruction, Conjuration, Blade and Light Armor. I'm ALL combat. Though that line of reasoning does make sense.
And yeah, you can stay level one and work until you've got 100 summoning, but since all you can summon is a Scamp (You won't have the Magicka to summon anything higher), which lasts 20 seconds and is REALLY useless, I'm not sure how that works.
Sepharo, move the diff slider down just a slight bit, you'll be AMAZED at how much easier the combats get. It does make a lot of sense if the slider is bugged, because it sure seems that way. It's like once you go to 50%, the game actually puts you at 500% or something.