Generally if the loan is being paid then the mortgagee cant cancel a loan. One nasty seen during GFC and even in recent times etc is a lender withdrawing from market.
One of problems with using fringe players in lending v's the majors can also result in them leaving the market. They can ask...