A few things to consider:
The trees which are causing the damage are in who's garden area?
Is the fence is on common property or within your area on title?
Do you have the right to replace the fence with anything you want ie who controls this fence (similar issue to venetian blinds, external...