Depends on your financials, your income stability, your risk and failure tolerance..... and ultimately what your end goals are.
If your reasoning is "mortgage broker said..." I'd be a bit unconfortable..... (unless they were very cluey).
There are different risk profiles for apartments...