Im confused about your confusion. It does work both ways. The issue, per the meme, is that someone may not have wanted to engage further and not state that they do not want to. By giving them your contact info they can think about it and then decide that it is in fact mutual and not feel pressured by the moment, so it still works as advice in that social situation.