Well,
From what I saw, for your $250 or whatever my friend paid for it, you get the PSP, and a UMD disk with a copy of Spiderman2 movie on it. I think it might also have contained the headphones/remote thingy as well.
On the software side it's got virtually nothing; view photos, view video, or listen to music. Like WOW! I can do all this stuff and much more on a PocketPC. It only accepts Sony memory stick (and even that is a "new" format, which is incompatible, without an adapter, with the "old" memory stick).
I've never been a fan of Sony's. I have owned an original Playstation, and I currently own a PS2. All the Sony gear that I look at (and I'm a bit of a gizmo nut), seems to be for the fashion conscious, rather than more functional. And it's not cheap stuff.
The PS2 will be the only piece of Sony gear in my house...
Like I said these are just my impressions, I wouldn't try to dissuade anyone from buying one, merely stating my opinion...