I am a tax accountant and I would recommend all of the above.
The IRS withholding calculator is very good (& a tad conservative. Gee, who'd guess the IRS would want you to withhold more than you need to?
). But I also understand people don't like to owe - so conservative is good.
It's a good calculator, even for the complex situation, if you can understand it.
Then, once it tells you how many exemptions, you can plug in your info at paycheck city and see what your paycheck will look like. You can then adjust accordingly, if you want to be more aggressive or conservative, and play around with what different withholding amounts do to your check.
But I just wanted to chime in that both are great resources.
If you use Turbo Tax, or any software, that may be even better. (Easier to understand anyway, in place of the IRS calculator).