Can't touch this because of a heritage listing or that because it's a "tree of significance". Your tree in the middle of suburbia means you are now responsible for providing key habitat to cockatoos or whatever else.
Whilst I agree partly with what you are saying, on the other hand I'm glad there are laws in place to protect certain things. Imagine the other extreme where we let people do whatever they like with 'their land', and let developers rip down all the beautiful old buildings and build what they like. We'd end up with no trees and heaps of cheap ugly concrete boxes one after the other, squeezing as much as possible onto the land for as little as possible, to make as big a profit as possible.