"While the existing fence was pretty bad, it still worked as a fence. Their cutting of their block has removed all our garden, scrappy as it was, and really, we could get stroppy that we were not informed."
Don't understand the 'removed all our garden". Have they entered your property to access...