I would listen to the wife. What happens when you're living in the townhouse and you decide to have another one in 3 years time and the townhouse hasn't appreciated enough to cover stamp duty?
If you are forced to sell in less than 5 years, you maybe up for a loss.
A more important factor...