I don't think you have much of a case either. You assumed the agent was over quoting when you entered the agreement. Would be far easier and safer and maybe even fairer to pay the amount owing - how much is it?
On the other hand I don't think they have any basis to lodge a caveat and if they...