Whoever wants it or needs it the most pays the most.
It's the universal law of negotiation.
But when buying with a big deposit, all the lenders want my business, so I get to pick & negotiate.
Though I gotta admit I've never worried about a few $$ here and there, I'd try to work out the long term...