Actually, I would argue that these days, a "modest" home is more likely to be an apartment/unit/townhouse rather than a detached house. And definitely with a mortgage.
Our first PPOR certainly will be (once it's built) as we decided it was more important for us to live close to work and...