Lack of game support. The obscene price of the memory sticks. The cost of the unit itself. The lockdown on the number of users per device (ie. . 1 user). Now the lockdown of 1 user to a memory stick so you can't share memory sticks. The hardware is great but every single other aspect of it fucking sucks and you are only hurting the industry by giving sony money for this pile of crap.
Only one handheld gaming unit has gotten it right imo... the ipod touch/iphone.