If I have to turn away potential buyers due to accepting the deposit, then I would keep the deposit.
But it would be a minimal amount - say $500.
I wouldn't allow any "subject to finance" clauses, but that's stated upfront and they have the choice whether they want to leave the deposit.