I'm perfectly fine with the size of my iPhone 5S. It's still small enough to handle easily and not take up much space in my pocket or on my side (I have a case that clips to my waistband).
I've used bigger phones and they just felt bulky to me, and I have big hands too. If I want a large screen for something, I'd rather use a tablet (like my iPad Air).
It's funny how for a long time phones were getting smaller and smaller and small meant good. Even during the early years of the modern smartphone hardly anyone wanted a large screen. The large screen ones didn't sell or if they did, the owner ended up returning it because of how less-portable it was. Now for some odd reason, more and more people want to have a freaking tablet as a phone. I mean sure video and the web is better on a large screen, but that means you have to carry around more bulk. That's not for me. If someone else doesn't mind that though, then go for it.