Don't forget - natural selection would have favoured those who were a little 'wary' of venturing off into the middle of the 'other, different looking tribe' to give them a hug and say hi. More often than not that would have ended with a rock to the head.
Obviously, we have this other thing...