I did a little research on podcasting, but I don’t really have any solid recommendations. It seems to boil down to budget first, then choosing items from there. It looks like the two main pieces of equipment are microphone and headphones, assuming a computer is already in use. For software there’s free stuff like Garage Band, Audacity, and Skype, and capture and editing programs from Rogue Amoeba. Of course, you can go crazy and spend thousands on really high end stuff, but I don’t think that’s necessary, especially just starting out. This equipment will work for other projects too, like video lectures/essays, voice overs, Google Hangouts, and so on.
The Blue Yeti mic is highly recommended, and it’s not too pricey at $119
– I’ve seen these as low as $79 on Amazon now and then (for Silver)
– The Yeti is much higher quality than the $50 Snowball (skip the Snowball)

The Rode Podcaster mic at $229
– It’s good, but the reviews are sort of all over the place, many swear by it, others go for more expensive stuff

There are other mics out there, but I like both of these. The Yeti comes with a table stand, the Rode Podcaster doesn’t but this table stand should work. You might need a USB cable or extension depending on the distance between the computer and the mic.

The Yeti has a mute button on it, but the Podcaster doesn’t (could be muted in software, I think). The Podcaster points right at you, while the Yeti has three settings (for you only, you and someone across, and 360 degree capture).

Headphones are a wild ride, anything will work really, but overall you’d want a closed-back, over-ear model, no noise canceling, and wired only. There’s a pretty good headphone review by Marco Arment for semi-expensive headphones ($300 range), but even basic earbuds would work. The Wirecutter has some headphone reviews in various price ranges, and their review of the $100 Sony MDR7506 is backed-up by positive comments on Amazon.

As a starter kit, the Blue Yeti mic & Sony MDR7506 headphones — $229

Rogue Amoeba’s Audio Hijack/Fission bundle — $60

Optionally, I’d add a boom arm for either mic, the Rode PSA1 — $99

For the Rode Podcaster, also the Rode PSM1 shock mount — $39

For the Blue Yeti, no shock mount (the Blue mount isn’t good, and no other ones work well, it can mount straight on the Rode arm or stay on the table).

I have the equipment already, but I’ve only used it for voice overs, not podcasting. I’ve never made a podcast, and I’m sure I’m forgetting something somewhere. I’ll need to look into the process of hosting the audio files, a website for the show notes (could be a free Tumblr account or something), a name for the podcast, possibly a domain name too.

Here’s some good links on the podcast process and equipment:
Jason Snell at sixcolors.com – Want to do a podcast? Don’t be intimidated & How I podcast: Recording & How I podcast: Editing
Casey Liss – How I Make Podcasts