If the lean into your side is severe, you could argue that it has to come down for safety reasons.
I am in Canberra and don't know whether the same applies where you are. I had the same issue, except the tree's roots and first two metres of the trunk were on my property, and thereafter in...