The key issue is - What the jurisdiction ? What does the deed say. which law governs the trust ?
Like all legal issues, this isnt a complex area providing you get legal advice. There can be serious issues - Gina Reinhart encountered this precise issue. And a few kids who grew tired of her...