This false assumption of you being in charge is the root cause of all of your troubles.
Once you let go of this ridiculous notion that as the Owner of the property you have any say in what / where / how anything is allowed to be built on your land, everything will be smooth sailing from there...