I am currently using a Zune 30 as my mp3 player. I would like to upgrade and am interested in Android, so I thought I'd just buy a used phone and use it for an mp3 player and apps (sort of like an Android iPod touch).
I imagine most of the used phones available are 2+ years old (if they were first bought with most carriers' 2 year obligation). For less you might find something without the phone function but otherwise more capable (and new.) I haven't shopped that market, but I know I've seen phone/touch sized android players in the stores, and have seen 7 inch android tablets for under $100 (not top performers but, again, compare to 2 year old phones...)