No I don't think you're right. You're buying based on potential - you don't necessarily need to be the person that realises that potential. For example if they rezoned a property to high rise, and you had an opportunity to get something early while everyone was unaware, would you pass up the chance to buy something that will go up 2x - 10x in a very short time frame? You may not have the capability or financing to build the high rise but you'll be damn well rewarded for identifying the opportunity before everyone else.
Yield is a different question entirely. Yield is about how you keep afloat in the mean time. It's no different from a low yield high CG property VS high yield low CG property. If you buy a property that's $1000k p/m cash flow negative after rent, what's the difference between that and a vacant block of land that had $0 rent but is still $1000k p/m cash flow negative?
And the end of the day it all comes down to value. Is a house good value? Is an empty block good value?