So you wouldn't recommend using a Mortgage Broker local to the area that you are trying to buy in? Like using a Ballarat broker for a Ballarat purchase, or a broker from Outer Melbourne East for Inner West Melbourne?
Obtaining finance is very similar across all states, so theres no need to find local brokers for each purchase. Unlike legals, where some states have specific requirements (qld for example is slightly different to NSW), finance is much more transferable from state to state. E.g. Commbank, NAB, Westpac, etc operate all across Australia and brokers around the country have similar access to the same banks.
If your looking for someone you can meet face to face and interact with, then someone close by is probably ideal. If your happy for online/phone contact, any broker in the country will do.
Generally swapping brokers for each purpose comes at a time/inconvenience cost. Quality brokers would also generally plan around your goals (not just for the purchase ahead, but for the next 3-5-10 years).
Personal view is to find a broker who does you a great job, that your happy with (perhaps shopping around initially to find this person) and then build a long term relationship with them.
I may be biased though, given my role. Although i'd guess most of the good brokers on the forums have a large majority of their clients being existing ones, or ones referred to from existing clients.