How'd you get the loan for the townhouse before you had a full time job?
In any case, best thing I would say is to keep saving. Look for opportunities. Lots of people are doing things in NSW (granny flats, mining towns etc) which may not be present in Victoria.