In New South Wales, the deposit is held between exchange and completion/settlement by the "stakeholder", which is usually the agent (or the vendor's solicitor in their trust account, if there is no agent).
More often than not, the vendor will include a special condition in the contract allowing them to have the deposit released to them if they need to pay a deposit or stamp duty on the next property they are purchasing. Purchase
There are risks with allowing the deposit to be released (e.g. if you have the right to pull out of the contract and get your deposit back, it's obviously a lot harder if it's no longer sitting in the agent's trust account), so purchasers should always ask for this special condition to be deleted.
As for delayed settlement ... In New South Wales, contracts usually specify that if either party cannot settle on the nominated settlement date then the other party has the right to serve them with a document called a Notice to Complete, which makes time "of the essence" (ie an essential term in the contract breach of which entitles the other party to terminate), and if the other party does not settle within 14 days then the non-defaulting party has the right to pull out of the Contract and get their deposit back (if they are the purchaser).
In NSW, penalty interest is usually only payable by the purchaser if they delay settlement. There is no such penalty payable by the vendor.
I'd be careful settling with the tenants in place - you risk them staying on at the property and being difficult to remove so that you can move in.
I would suggest delaying settlement until the tenants move out. I would nevertheless serve on them a Notice to Complete to protect your rights, just in case you decide that you no longer want to proceed with the purchase.