Sounds like a good idea in theory, but the deposit is forfeited if you breach the contract by not settling. So if, for example, your bank isn't ready to settle on settlement day, you lose your deposit. Better to lose $1,000 or $5,000 than (say) $30,000.
I understand about not trusting anybody...