Concentration risk is one reason and serviceability (or lack of) is another.
Your point about rates is a valid one but rate is not the first factor on the list. If your current lender says "sorry, no more money for you" due to hitting the serviceability ceiling and another lender rolls out...