I think you should ask your accountant to clarify. A better use of the offset money is to use it to pay off a bit of the PPOR loan, then redraw it out straight away to pay for the 20% deposit on a new IP.
Credit hits are an issue because it all counts towards credit scoring. If you have huge net assets and income with low LVRs then it is less of an issue but if you are going for LMI etc it is a lot more critical for getting finance.