You don't say where the house is, but I am in Brisbane, and if this was my problem, I would follow through with the breach notice. If you have not put anything in writing about what you plan to "improve" then he has no case.
I would stay professional, follow through and if needed, take him to...