Go to the irs.gov website
here. The W4 has a worksheet that will walk you through how many withholdings. Last year we claimed 9 and still received a refund. Those child tax credits make it hard to have not enough withheld. We have two children, too. Our income was about $55k taxable.
Don't figure in the non-taxable income. It is not calculated in your taxes.