Used to do it all the time when I worked on the development side of things.
Push the fence a bit, and put a nice new shiny colorbond fence up. Doesn't break any rules, as it is within your boundary, you don't touch the asbestos nor pay for its removal. Everyone (well, mostly you) wins...