If they're not returning your call, or doing what they should do - why bother paying them anymore?
Changing agents isn't a big deal, and fortunately for VIC there isn't really any notice period - usually the new agent is who notifies the old agent.
Also, a lot to consider when changing...