eBay is your friend, you can get batteries for ancient phones like the Samsung Intercept for $4 still, brand new. Luckily for us, batteries are a highly commodified component, you can replace the cells yourself with minimal effort if you so choose.

When I'm feeling like an uber-cheapskate I rebuild laptop batteries, recycling the failed cells and replacing them with good cells from other batteries.